Google Glass helps kids with autism understand faces

Stanford-developed app makes a game of recognizing facial expressions

Alex took part in a pilot study in which a smartphone app paired with Google Glass was shown to help children with autism understand emotions conveyed in facial expressions. 
Steve Fisch

Last year, a few weeks after he began participating in a Stanford pilot study of a novel autism treatment, a boy named Alex did something that really surprised his mother.

“He was engaging with my eyes, flickering his eyes at me,” said Donji Cullenbine, whose son is now 9. It was a big change for Alex, who had always felt anxious about looking at people’s faces and had struggled to understand what others might be thinking.

The pilot study demonstrated that Google Glass technology could help kids with autism recognize facial expressions. Stanford researchers devised an app that pairs with Google Glass to act as an emotion-recognition tutor for children with autism. After one to three months of regular use, parents reported that children in the study made more eye contact and related better to others.

“Across the country, we have too few autism practitioners,” said the study’s senior author, Dennis Wall, PhD, associate professor of pediatrics and of biomedical data science. Early autism therapy has been shown to be particularly effective, but many children aren’t treated quickly enough to get the maximum benefit, he said. “The only way to break through the problem is to create reliable, home-based treatment systems. It’s a really important unmet need.”

Learning game

Autism is a developmental disorder that affects 1 in 59 children in the United States, with a higher prevalence in boys. It is characterized by social and communication deficits and repetitive behaviors.

Before her family participated in the study, Cullenbine tried using gentle encouragement to boost Alex’s social interactions. “I would smile and say things like, ‘You looked at me three times today!’ But it didn’t really move the bar,” she said. Using Google Glass transformed how Alex felt about looking at faces, Cullenbine said. “It was a game environment in which he wanted to win — he wanted to guess right — and he got an instant reward when he did.”

‘Superpower Glass’

The researchers named the new therapy “Superpower Glass” to make it appealing to children. The therapy is based on applied behavior analysis, a well-studied autism treatment in which a clinician teaches emotion recognition using structured exercises such as flash cards depicting faces with different emotions. Although traditional applied behavior analysis helps children with autism, it has limitations: It must be delivered one on one by trained therapists, flash cards can’t always capture the full range of human emotion, and children may struggle to transfer what they learn to their daily lives.

The Google Glass device, which is linked with a smartphone through a local wireless network, consists of a glasses-like frame equipped with a camera to record the wearer’s field of view, as well as a small screen and a speaker to give the wearer visual and audio information. During the “Superpower Glass” therapy, as the child interacts with others, the app identifies and names their emotions through the Google Glass speaker or screen.

He told me, ‘Mommy, I can read minds!’

The app recognizes eight facial expressions and can be used in three modes: free play, in which the device names the expression of anyone in the wearer’s field of view, and two game modes that provide fun, rewarding ways for children to practice naming the facial expressions. The study included 14 families, each of whom had a child between ages 3 and 17 with a clinically confirmed autism diagnosis.

At the end of the study, the children’s scores on a social-skills questionnaire indicated less-severe autism symptoms. Six participants had experienced changes in their scores large enough to move down one step in the severity of their autism classification, and 12 of the participating families reported that their kids made more eye contact.

Marked improvement

Parents’ comments in interviews with the researchers helped illustrate the improvements, Wall said. “Parents said things like ‘A switch has been flipped; my child is looking at me.’ Or ‘Suddenly the teacher is telling me that my child is engaging in the classroom.’ It was really heartwarming and super-encouraging for us to hear.”

A few weeks into the trial, Alex began to realize that people’s faces hold clues to their feelings. “He told me, ‘Mommy, I can read minds!’” Cullenbine said. “My heart sang. I’d like other parents to have the same experience.”

Wall’s research team is now completing a larger, randomized trial of the therapy. They also plan to test it in children who have just been diagnosed with autism and are on a waiting list for treatment. The project’s website has more information about ongoing clinical trials for those interested in participating in the research.