Resources

Below is a collection of tutorials, templates, and "best practices'' guidelines to aid in the conduct of research rigor and reproducibility (R&R), as well as resources on open science, data management and similar initiatives with overlapping topics. 

This list of resources is also intended to serve as a library from which your research team may select appropriate tools to structure and manage aspects of R&R relevant to your own research.

We are currently in the process of redesigning the website and adding additional resources. If you have suggestions on resources we should include please contact us at https://med.stanford.edu/sporr/contact.html. 

NIH Rigor and Reproducibility Resources:

• Training Modules - National Institute of Health (NIH) training modules include videos and discussion materials, and focus on integral aspects of research rigor and reproducibility, such as bias, blinding and exclusion criteria. The modules are not comprehensive but rather provide an overview and a way to stimulate conversations.
• Frequently Asked Questions - Cover response to inquiries on policies and programs affecting the grants process
• Guidance: Rigor and Reproducibility in Grant Applications -  Learn how to address rigor and reproducibility in your grant application and discover what reviewers are looking for as they evaluate the application for scientific merit.
• NLM Tools for the Research Data Lifecycle -  email class - covering data sharing, data analysis, and research dissemination. Sign up to receive a short lesson via email every few days, over the course of a couple of weeks.

Rockefeller University - R3 Talk Series: Enhancing Scientific Rigor, Reproducibility, and Reporting -  includes topics: Introduction to Reproducibility, Data Management for Biomedical Research, Managing Experimental Protocols with protocols.io, Retractions, Elements of Statistical Power Analysis, Transcriptomics in Translational Science 

Heidi Seibold -  6 Steps Towards Reproducible Research

Reproducibility for Everyone (R4E) modules

• Introduction to Reproducibility, Data Management, Electronic Lab Notebooks, Protocol Sharing, Reagent Sharing, Bioinformatics, Code and Data Sharing, Data visualization, Publishing

Children’s Hospital of Philadelphia - Data and Analytics for Research Training (DART):

• Reproducibility, Generalizability, and Reuse module 
• Computational modules

Guide for Reproducible Research (The Turing Way)

• Open Research, Version Control, Licensing, Research Data Management, Reproducible Environments , BinderHub, Code quality, Code Testing, Code Reviewing Process, Reusable Code, Continuous Integration, Reproducible Research with Make, Research Compendia, Risk Assessment, Case Studies

Library of Guidance for Health Scientists (LIGHTS) - searchable database for methods guidance articles

• Article describing the resource   

SPORR courses and trainings in Rigor and Reproducibility

The National Aeronautics and Space Administration (NASA) Year of Open Science resources

• Open Science 101 modules:
  • Ethos of Open Science
  • Open Tools and Resources
  • Open Data
  • Open Software
  • Open Results

Center of Open Science (COS) Resources 

• Openness and Reproducibility Research Practices Training (Paid service)
• Open Science Framework
• Preregistration initiative
• Register reports initiative

Guides on Open Science

• Passport for Open Science – A Practical Guide for PhD Students (2022 French Guide)
• Open Science: A Practical Guide for PhD students (2022 UK Guide)
• Open Science: A Practical Guide for Early-Career Researchers (2023 Dutch Guide)

Open Science Textbooks

Experimentology - An Open Science Approach to Experimental Psychology Methods

Open Science Training Handbook

Open Science Workshop Materials of the LMU Open Science Center

Other

Open Science Reading Group at Lane Medical Library - Learn about open science and discuss the application of open science practices in a biomedical context together with other members of the Stanford Medicine community.

Open Science Map of Resources from Around the World by Johanna Havemannd

National Institute of Health DMP resources

• NIH DMP sample plans (13 plans/templates)    

CURATED Training Modules

• C: Check Your Data
• U: Understand Your Data
• R: Request Missing Information
• A: Augment the Data Deposit
• T: Transform File Formats
• E: Evaluate the Overall Data Package 
• D: Document for Curation

Columbia University ReaDI Program - Resources and materials collected at the Research and Data Integrity (ReaDI) program at Columbia University, with special emphasis on data management.

Lab Manuals can be used to share the vision, goals, policies, as well as methods of producing rigorous and reproducible science. 

In 2022, Andreev et al. published a guide to help onboarding of new lab members - Research Culture: Welcome to the lab

In 2023, Kovacs et al. published a preprint with a lab manual template: Lab manuals for efficient and high quality science in a happy and safe work environment

We recommend checking out the following lab manuals: 

Benjamin-Chung Lab Manual (Stanford)

John Boothroyd Manual (Stanford)

Fraser Lab (UCSF)

Decision Lab Handbook (UCSF)

Additionally, the following guides might also be of use:

Strand, J. F. (2023). Error tight: Exercises for lab groups to prevent research mistakes. Psychological Methods.

Authorship Agreements explained by ORI (Office of Research Integrity) - PDF - Video

Authorship Agreement Template (developed by ScholCommLab) and Guide

Authorship Agreement Template (developed by Bay Area Open Science Group)

Tenzig App - App to help document CRediT taxonomy for publications