Individual Seed Projects

Mining Digital Life for Precision Prediction, Prevention & Early Detection

Byron Reeves, PhD

Paul C. Edwards Professor of
(650) 725-3033
Associate Director, Center for Policy, Outcomes and Prevention, Children's Health Initiative, Department of Pediatrics (2005 - 2014)
(650) 723-5331
Member, Biomedical Library and Informatics Research Commitee Study Section (NIH) (2002 - 2005)
(650) 725-0659

Nilam Ram, PhD

Professor, Human Development
and Family Studies, and Psychology
(814) 865-7038

  • Much of life is now digitized on personal screens including information about relationships, money, work, play, travel and much more. 
  • We are recording screenshots of digital personal life from laptops and smartphones, and identifying features related to health outcomes.
  • Screenomics is science of studying digital life experiences, exposures and actions through personal screenshots (the screenome) for their impact on thoughts, emotions, behaviors and health.
  • The screenshots that make up the screenome are recorded every 5 seconds that devices are turned on.
  • The images are encrypted, compressed and transmitted to Stanford servers for analysis.
  • We extract text and images from the screenshots and create searchable databases.
  • We currently have over 10M screenshots from ~400 people

  • We visualize the information obtained in the screenome and analyze the behavioral sequences for content and action patterns.
  • Examples above:
  • The color bars on the left are 1-week screenomes for 30 people; colors indicate the timing and material viewed.
  • The circle on the right shows switching across devices and among multiple types of content.  
  • Using machine learning and individual dynamic modeling, we search for the screenshot features and screenome actions, and combinations of them, that predict health outcomes. 
  • Examples above (from left):
  • Person searching for information about a suicide bridge.
  • Texting about bruises that won’t heal.
  • Exhibiting memory problems in communication with a spouse.
  • Asking about dark skin on the back of the neck and why it won’t wash off.