Frequently Asked Questions
In recent years, a variety of problems afflicting the reliability of biomedical research have received widespread attention, under the rubric of research reproducibility or replication. The ability to reproduce research is applicable to its methods, results, and inferences.
A study has good methods reproducibility if it provides enough detail to allow for the implementation – as identically as possible – of its experimental and computational procedures. The term reproducibility is sometimes wrongly equated with only computational reproducibility - the ability to obtain the same (statistical) results by re-running the analysis, or re-using the (deposited) code on the researcher's data. This approach fails to capture the complexity of re-running the design and data gathering and cleaning processes.
Results reproducibility refers to how well results are corroborated in a new study, in which the researchers have matched the original study as exactly as possible. Sometimes this is also referred to as replication.
Having obtained identical results, will two separate research teams draw the same conclusions? Not necessarily. Inferential reproducibility is a term for how much agreement there is between the conclusions stemming from an independent replication of a study or a reanalysis of the original study.
It follows that results reproducibility is a prerequisite for inferential reproducibility – and that both of these rely on the reproducibility of methods.