Workshop in Biostatistics

DATE: December 3, 2015
TIME: 1:30 - 2:50 pm
LOCATION: Medical School Office Building, Rm x303
TITLE: Reading a Couple of Million Papers with P-values
SPEAKER: John Ioannidis
C. F. Rehnborg Professor in Disease Prevention in the School of Medicine and Professor of Health Research and Policy (Epidemiology) and, by courtesy, of Statistics

 

Abstract:
Current progress allows the large-scale appraisal of large segments of the scientific literature in efficient meta-research evaluations that can include anywhere from hundreds to millions of published papers. One interesting application is the evaluation of large-scale use and reporting of statistical methods, statistical inference tools and effect sizes. For example, such assessments show that 97% of all abstracts that report P-values in biomedicine have P-values that are significant at the P=0.05 level and the same applies to full-text articles. One can also study the use and reporting of different effect sizes, measures of uncertainty (e.g. confidence intervals), Bayes factors, FDR, qualitative statements about significance, etc. The distributions of P-values and effect sizes in the literature and their evolution over time in meta-analyses and meta-meta-analyses can allow for some interesting insights about how scientific results are generated and (usually very selectively) reported. Modeling can try to address the extent of bias and other forces that shape why specific types of results are preferentially being reported. Empirical data will be discussed from various disciplines in the biomedical sciences and beyond.

Suggested readings:
Ioannidis JP, Fanelli D, Dunne DD, Goodman SN. Meta-research: Evaluation and Improvement of Research Methods and Practices. PLoS Biol. 2015 Oct 2;13(10):e1002264.

Serghiou S, Patel CJ, Tan YY, Koay P, Ioannidis JP. Field-wide meta-analyses of observational associations can map selective availability of risk factors and the impact of model specifications. J Clin Epidemiol. 2015 Sep 25. pii:S0895-4356(15)00423-0.

Patel CJ, Burford B, Ioannidis JP. Assessment of vibration of effects due to model specification can demonstrate the instability of observational associations. J Clin Epidemiol. 2015 Sep;68(9):1046-58.

Ioannidis JP. How to make more published research true. PLoS Med. 2014 Oct 21;11(10):e1001747.

Pereira TV, Horwitz RI, Ioannidis JP. Empirical evaluation of very large treatment effects of medical interventions. JAMA. 2012 Oct 24;308(16):1676-84.