March 28, 2011 - By Krista Conger
Apples to Apples is more than just a popular card game. It’s an important concept when comparing the results of published scientific studies. It’s impossible to draw accurate conclusions, for example, without an adequate description of a study’s design, the eligibility criteria for study participants or the identification of the statistical methods used.
“In the absence of guidance, there is wide variability in what investigators report and what they forget to report about their studies,” said study design expert and chief of the Stanford Prevention Research Center, John Ioannidis, MD, DSc. “It is important to make sure that all the essential features about the methods, design and results of a study are transparently available.”
To ensure consistency, many types of research studies have guidelines specifying how research results should be reported in scientific journals. In March, Ioannidis along with Mark Hlatky, MD, professor of health research and policy and of cardiovascular medicine, and a group of risk prediction researchers, epidemiologists, geneticists and others sponsored by the Human Genome Epidemiology Network published a similar set of guidelines for one of the hottest scientific and clinical topics today: genetic risk prediction studies. These studies seek to correlate specific DNA sequences with the likelihood that a carrier will develop particular disorders or conditions, and are a key component of what’s known as “personalized medicine.”
A statement about the GRIPS guidelines (for Genetic RIsk Prediction Studies) was recently published simultaneously in 10 journals: the Public Library of Science Medicine; Annals of Internal Medicine; British Medical Journal; Circulation: Cardiovascular Genetics; European Journal of Clinical Investigation; European Journal of Epidemiology; European Journal of Human Genetics; Genetics in Medicine; Genome Medicine; and Journal of Clinical Epidemiology.
“Adhering to these guidelines can be helpful in making the research process more transparent, complete and accurate,” said Ioannidis, who is known for pointing out flaws in many published studies. “This enhances the chances that the results and their interpretation are more appropriate and balanced.”
Much of Ioannidis own work involves strengthening the way that research is planned, carried out and reported. He outlined some of the problems he observed in a 2005 essay in PLoS-Medicine titled, “Why most published research findings are false.” The essay remains the most-downloaded article in the history of the Public Library of Science, according to the journal’s media relations office.
For instance, Ioannidis noted that until five years ago, many studies that linked a genetic variant to a specific disease were often later proven wrong. In some cases, it was because the number of samples used in a study was too small; in others, the way the data were reported skewed the results.
The guidelines don’t tell a researcher how to design a study; only how it should be reported in the scientific literature. “Investigators are free to do as they wish and to follow any type of new innovative design and analysis they feel is preferable for the risk model they have in mind,” said Ioannidis. “It is important, however, for them to record for other researchers what they have done. Genetic risk prediction could have a major impact on personalized medicine. We want to have the best studies informing us about its potential.”
However, the guidelines are not set in stone. They can be changed in the future, or used to guide the direction of future research, according to Ioannidis.
“They specifically give suggestions for careful discussion of the evidence and its limitations. This leads naturally to improving the chances of rational planning as to where we go from here when designing future research. In all, more accuracy and transparency may decrease the dissemination of inflated promises and false-positive claims.”
About Stanford Medicine
Stanford Medicine is an integrated academic health system comprising the Stanford School of Medicine and adult and pediatric health care delivery systems. Together, they harness the full potential of biomedicine through collaborative research, education and clinical care for patients. For more information, please visit med.stanford.edu.