Stanford launches center to strengthen quality of scientific research worldwide

Norbert von der Groeben

Steven Goodman (left) and John Ioannidis will direct a new center focused on identifying weaknesses in the way scientific research is conducted and offering methods for improvement.

A new center at Stanford University aims to transform research practices to improve the reproducibility, efficiency and quality of scientific investigations.

Scholars at the Meta-Research Innovation Center, or METRICS, will focus on conducting research about research. Their mission: to promote excellence in research through collaborations around the world. The center’s launch has been made possible through a $6 million grant from the Laura and John Arnold Foundation.

The center will be co-directed by John Ioannidis, MD, DSc, professor of medicine and of health research and policy and director of the Stanford Prevention Research Center, and Steven Goodman, MD, MHS, PhD, professor of medicine and of health research and policy and associate dean for clinical and translational research at the School of Medicine.

“Scientists have made amazing discoveries to date, but we have an opportunity to accelerate advances,” said Ioannidis, who is also the C.F. Rehnborg Professor in Disease Prevention. “Our center is taking a methodical approach to identifying weaknesses in the research enterprise and evaluating ways to enhance its efficiency. We will be looking for ways to reduce biases in study design, data interpretation and outcomes reporting.”

The problems with research

Ioannidis has devoted much of his career to empirically examining the biases and the reproducibility of published science. (Reproducibility, meaning that an experiment can be reproduced with the same results, is key to validating and building on scientific findings.) Ioannidis is one of the most highly cited scientists of his generation in the scientific literature, according to Microsoft Academic rankings. He is best known for his 2005 PLoS Medicine paper “Why most published research findings are false,” which is the most accessed and downloaded article in the history of Public Library of Science, with more than 1 million views to date.

Goodman’s research is focused on scientific inference, the process of using statistics to draw conclusions from data in the presence of uncertainty. He has spent much of his career trying to improve the accuracy and transparency of scientific research. He is the senior statistical editor of the Annals of Internal Medicine, vice-chair of the methodology committee of the federal Patient-Centered Outcomes Research Institute and, for the past decade, editor-in-chief of the journal Clinical Trials.

“All scientists share the common goals of searching for truth and of improving their methods,” said Goodman. “But it is sometimes difficult to know which research and reporting practices actually lead to more reliable conclusions without studying research itself, the domain of the meta-research.”

Recent public concern about research credibility has prompted the government to take action. On Jan. 27, Francis Collins, MD, PhD, director of the National Institutes of Health, announced a series of initiatives to enhance NIH-supported research reproducibility.

METRICS plans to work closely with researchers and policymakers on this issue. As a first step, the center named Deborah Zarin, MD, a senior scientist at the NIH’s National Library of Medicine, as its first visiting scholar. In her role as director of ClinicalTrials.gov, the largest public registry and results database of publicly and privately funded clinical studies conducted around the world, Zarin studies the quality of trial reporting and analysis in support of evidence-based clinical and policy decision-making.

“I’m excited about collaborating with colleagues at METRICS in conducting research aimed at improving trial reporting,” said Zarin. “It’s part of our promise to research participants, who put themselves at risk, to make optimal use of these data in order to advance medical knowledge and, ultimately, improve medical care.”

Spanning the sciences

METRICS’ core group of interdisciplinary scholars will be working on various aspects of meta-research, from methodologies to processes to policy. Because these research principles apply to all scientific research, not just medical research, METRICS supports all Stanford disciplines. The center is managed under Spectrum, an independent Stanford center funded in part by an NIH Clinical and Translational Science Award. Spectrum is directed by Harry Greenberg, MD, senior associate dean for research at the medical school, and overseen by Ann Arvin, MD, dean of research and vice provost at Stanford.

“METRICS will engage in the very important task of helping biomedical researchers enhance the reliability of their work,” said Arvin. “The program’s initiatives to improve approaches to study design and data interpretation, and to optimize the accuracy and efficiency of this research, will substantially accelerate the transfer of scientific knowledge for the benefit of society.”

Lloyd Minor, MD, dean of the School of Medicine, said, “If we are to seize upon the promises of this century of biomedicine, we must ensure that our research is replicable and reliable. John and Steve are leaders in the nascent field of meta-research, and we are most grateful for the support of the Laura and John Arnold Foundation in furthering their work to promote accuracy in the design of research studies and in the interpretation and the reporting of results.”

METRICS will also provide educational funding for students and scholars; organize collaborative working groups that include academics, policymakers, research funders and the public; and help establish similar initiatives worldwide.

“There is widespread concern about research reliability right now, but the best, practicable solutions are not obvious, requiring changes both in methods and in incentive structures,” said Goodman. “We want METRICS to be a place where scientists come together to develop and help implement those solutions, and we invite any member of the Stanford community to join us in working on these issues.”