2017 Individual Seed Projects
Mining Digital Life for Precision Prediction, Prevention & Early Detection
- Much of life is now digitized on personal screens including information about relationships, money, work, play, travel and much more.
- We are recording screenshots of digital personal life from laptops and smartphones, and identifying features related to health outcomes.
- Screenomics is science of studying digital life experiences, exposures and actions through personal screenshots (the screenome) for their impact on thoughts, emotions, behaviors and health.
- The screenshots that make up the screenome are recorded every 5 seconds that devices are turned on.
- The images are encrypted, compressed and transmitted to Stanford servers for analysis.
- We extract text and images from the screenshots and create searchable databases.
- We currently have over 10M screenshots from ~400 people
- We visualize the information obtained in the screenome and analyze the behavioral sequences for content and action patterns.
- Examples above:
- The color bars on the left are 1-week screenomes for 30 people; colors indicate the timing and material viewed.
- The circle on the right shows switching across devices and among multiple types of content.
- Using machine learning and individual dynamic modeling, we search for the screenshot features and screenome actions, and combinations of them, that predict health outcomes.
- Examples above (from left):
- Person searching for information about a suicide bridge.
- Texting about bruises that won’t heal.
- Exhibiting memory problems in communication with a spouse.
- Asking about dark skin on the back of the neck and why it won’t wash off.