Resources

Below is a collection of tutorials, templates, and "best practices'' guidelines to aid in the conduct of research rigor and reproducibility (R&R), as well as resources on open science, data managment and similar initiatives with overlaping topics. 

This list of resources is also intended to serve as a library from which your research team may select appropriate tools to structure and manage aspects of R&R relevant to your own research.

We are currently in the process of redisinging the website and adding additional resources. If you have suggestions on researces we should include please contact us at https://med.stanford.edu/sporr/contact.html

Rigor and Reproducibility

NIH Rigor and Reproducibility Resources

  • Training Modules - National Institute of Health (NIH) training modules include videos and discussion materials, and focus on integral aspects of research rigor and reproducibility, such as bias, blinding and exclusion criteria. The modules are not comprehensive but rather provide an overview and a way to stimulate conversations.
  • Frequently Asked Questions - Cover response to inquiries on policies and programs affecting the grants process
  • Guidance: Rigor and Reproducibility in Grant Applications -  Learn how to address rigor and reproducibility in your grant application and discover what reviewers are looking for as they evaluate the application for scientific merit.

 

Reproducibility Textbooks

Heidi Seibold -  6 Steps Towards Reproducible Research

 

Reproducibility for Everyone (R4E) modules

  • Introduction to Reproducibility, Data Management,  Electronic Lab Notebooks,  Protocol Sharing, Reagent Sharing ,Bioinformatics, Code and data sharing, Data visualisation, Publishing

 

Data and Analytics for Research Training (DART) -  Children’s Hospital of Philadelphia:

Reproducibility, Generalizability, and Reuse module 

Computational modules

 

Guide for Reproducible Research (Turning way)

  • Open Research, Version Control, Licensing, Research Data Management, Reproducible Environments , BinderHub, Code quality, Code Testing, Code Reviewing Process, Reusable Code, Continuous Integration, Reproducible Research with Make, Research Compendia, Risk Assessment, Case Studies

 

Library of Guidance for Health Scientists (LIGHTS) - searchable database for methods guidance articles

 

SPORR courses and trainings in Rigor and Reproducibility

Open Science

Data Managment and Data Management Plans

National Institute of Health DMP resources

 

CURATED Training Modules

  • C: Check Your Data
  • U: Understand Your Data
  • R: Request Missing Information
  • A: Augment the Data Deposit
  • T: Transform File Formats
  • E: Evaluate the Overall Data Package 
  • D: Document for Curation

 

Columbia University ReaDI Program - Resources and materials collected at the Research and Data Integrity (ReaDI) program at Columbia University, with special emphasis on data management.

Lab Manual (Templates)

Lab Manuals can be used to share the vision, goals, policies, as well as methods of producing rigorous and reproducible science. 

In 2022, Andreev et al. published a guide to help onboarding of new lab members - Research Culture: Welcome to the lab

In 2023, Kovacs et al. published a preprint with a lab manual template: Lab manuals for efficient and high quality science in a happy and safe work environment

We reccomened checking out the following lab manuals: 

Benjamin-Chung Lab Manual (Stanford)

John Boothroyd Manual (Stanford)

Fraser Lab (UCSF)

Decision Lab Handbook (UCSF)

Additionally, the following guides might also be of use:

Strand, J. F. (2023). Error tight: Exercises for lab groups to prevent research mistakesPsychological Methods.