Merging silos: MCHRI awardee assembles transdisciplinary team and combines data sources to track children’s activity

September 1, 2020

By Laura Hedli

If someone goes for a run free of a phone, Fitbit, or other device that records activity, did it really happen? Well, it depends on how you look at it.

As medicine begins to rely more heavily on technology to measure health, Professor of Medicine and of Biomedical Data Science, Manisha Desai, PhD, is researching optimal approaches for analyzing data collected from wearable devices, otherwise known as accelerometry data.

Today, gadgets and gizmos aplenty track our movements to an unprecedented degree. In fact, a typical smartphone can record activity up to 100 times per second. The challenge becomes then how to make sense of such busy and often incomplete data.

Dr. Desai, who is the Director of the Quantitative Sciences Unit, assembled a community of researchers from the Schools of Medicine and Engineering with diverse backgrounds including biostatistics, bioinformatics, pediatric medicine, sleep medicine, public health, computer science, bioengineering, and mechanical engineering. In 2015, the team received a Transdisciplinary Initiatives Program (TIP) award of $200,000 from the Stanford Maternal and Child Health Research Institute (MCHRI); Dr. Desai was the Principal Investigator. She and her colleagues proposed studying and applying the strengths of their respective disciplines in order to find the most effective methodologies for measuring children’s physical activity and sleep.

Such enormous amounts of mobile device data have outpaced the research community’s ability to interpret them. Traditional methodologies have relied on crude cut-points based on step count to group physical activity level into categories like “vigorous,” “moderate,” or “light.” Dr. Desai and her team wanted to take a more nuanced approach by assessing the timing and frequency of activity in order to understand how patterns of activity may affect outcomes like body mass index (BMI) or cholesterol.

“We saw a real sparsity in the literature and a real need for new methods,” Dr. Desai says.

The TIP award was the genesis for the team’s novel mobile health data research and propelled Dr. Desai to apply for funding from the National Institutes of Health. In 2018, after receiving a strong score that ultimately did not result in a fundable award, she applied for and received a MCHRI Bridge Support award for $100,000.

The Bridge Award allowed the team to retool their aims and complete preliminary findings to develop a new proposal for a grant sponsored by the National Library of Medicine. Last month, Dr. Desai received news that the proposal received a score in the 2^nd percentile. She is hopeful she will receive the award for a start date this fall.

A tale of two datasets

Prior to applying for the TIP Award, Dr. Desai had been collaborating with Professor of Medicine Tom Robinson, MD, MPH, on a multi-center randomized controlled trial to reduce obesity in children. Known as the Stanford GOALS study, the intervention took place over several years where 260 children, ages 7-11, were asked to wear accelerometers for one week annually. Meanwhile, the research team collected physiologic, behavioral, and psychosocial data on the participants.

“It is a really rich data source,” Dr. Desai remarks of the GOALS study. “Here we were actually recording their movements, and in a continuous and real-time uncontrolled environment. It’s a longitudinal dataset with time-series snapshots of physical activity.”

Drs. Desai and Robinson thought the GOALS dataset would be ideal for testing methodologies; however, for all its real-world strengths there were critical problems the researchers had to tackle before they could begin to correlate children’s patterns of behavior with health outcomes.

First, even though they had asked the children to wear the accelerometers for a week each year, sometimes participants only wore the device for a day or three out of the seven requested. How would the researchers account then for non-wear time?

Second, the accelerometer was always recording even if it wasn’t being worn. So, the signal the accelerometer recorded when a child was relaxing on the couch looked very similar to the signal recorded when the device was sitting in a drawer. How would they distinguish missing data from sedentary activity?

For comparison, they TIP Award team used another dataset with no missing data and chock full of recordings of sedentary activity. The Stanford Accelerometer Sleep Study, collected by Dr. Robinson and Professor of Psychiatry and Behavioral Sciences Clete Kushida, MD, PhD, represents data from 200 children, ages 2-17.

Accelerometers recorded the children’s movements during an overnight polysomnography at Stanford’s Sleep Medicine Center. Wake and sleep time, active-wear and non-wear time, as well as the five stages of sleep have all been verified using polysomnographic recordings and video observation. While the Stanford Accelerometer Sleep Study is a limited dataset, its strength is its completeness and inclusion of true labels of non-wear time.

An ensemble approach

Given the transdisciplinary nature of the research, the team met regularly to learn about best practice methodologies that are used in different fields to assess large amounts of data with lots of noise. Experts in each field led the discussions, and two postdocs tracked the team’s progress across disciplines.

Together, the researchers made new discoveries. For example, they found that combining machine learning algorithms, known as an ensemble approach, performed better at classifying non-wear periods than any one individual machine learning or statistical method. In addition, some machine learnings methods, like hidden Markov Models, performed worse than they had anticipated, but still proved useful in improving predictions when included in the ensemble.

“An ensemble approach says let’s exploit information from multiple learners because we don’t know exactly which method or learner is going to do the best,” Dr. Desai explains. “If our goal is to predict whether this state I’m in is non-wear or sedentary behavior, I’m going to borrow strength across multiple learners in some fashion.”

In addition, multiple imputation methods can be effective in filling in the missing pieces about non-wear periods, but how to apply such methods is not straightforward in the presence of time-series data sampled at such high frequencies. The researchers found that it was preferable to include data from all days and participants rather than tossing out incomplete recordings of days with limited hours or from participants with only a few complete days of wear. It’s standard practice to exclude partial information, but multiple imputation methods allowed for the inclusion of more data and improved upon the bias introduced by exclusion.

The team is now working with a third dataset – a simulated dataset – to test their ensemble methodology as well as their missing data methods. This simulated dataset marks the beginning of the next step of the project, which aims to define what patterns of physical activity correlate with outcomes like reductions in body mass index (BMI).

The researchers will generate a complete simulated accelerometer dataset for hypothetical persons that captures a key relationship: For example, more physical activity = lower BMI. Then, they will introduce random, expected noise into the dataset and eliminate certain data to mimic missing information in a real-world dataset. Given the investigators know the true relationship between physical activity and BMI in the simulated data, they can evaluate the performance of various methods by recording and comparing how well the methods capture that known relationship.

We’re all in our little world working on our own thing. It’s so surprising to come out of that world and see that other people are working on similar topics, but from a different perspective.

How will it work in the real world

Dr. Desai and her team have logged five years of work and are just now working on clustering patterns of activity using children’s baseline data from the GOALS study. Their ongoing efforts underscore how challenging it is to measure mobile health data in a meaningful way.

“I think it brings up really terrific opportunities having so much data, but I think that we have to be really careful about how we process and analyze this data,” Dr. Desai says. The future of medicine, she suspects, might involve providers routinely grabbing data from a patient’s device and informing and intervening based on what they observe.

To that end, the researchers are working on making open source software available to the public for analyzing accelerometer data. Others will be able to learn from examples and build upon the ensemble methodology to suit specific needs. Money from the MCHRI TIP award supported initial development of the software.

Dr. Desai still meets weekly with the researchers on the project, crediting MCHRI with giving her the time and means to learn about the importance of integrating ideas from computer science, engineering, physical activity, and pediatric medicine into methods development.

“It’s definitely prevalent at Stanford to be siloed,” Dr. Desai says. “We’re all in our little world working on our own thing. It’s so surprising to come out of that world and see that other people are working on similar topics, but from a different perspective.”

The MCHRI TIP award represents an opportunity to join different perspectives into a unified and new approach to tackling a particular problem. Dr. Desai is convinced that the transdisciplinary approach will result in a set of tools for accelerometer studies that is much improved over what would have been created from investigators coming from the same discipline.

Laura Hedli is a writer for the Division of Neonatal and Developmental Medicine in the Department of Pediatrics and contributes stories to the Stanford Maternal and Child Health Research Institute.