We have four key areas of study: sensorless sensing, subtle interventions, mindful commute, and conversational agents. Our projects aim to design not only useful and efficient wellbeing interventions, but also passive behavioral and affective sensors.
One way to make health tehcnology “disappear” in the environment is to not even install any hardware, but rather to obtain biomarkers from existing everyday devices. By combining human-computer interaction (HCI), digital signal processing (DSP), and neuro-biomechanics, we are creating biomechanical models of muscle stiffness in the arm, correlated with mental stress, by repurposing data from a computer mouse or a car steering wheel – a method that we provocatively call “sensorless sensing". This work has two contributions to science and design: 1) we enable researchers to explore stress at a large scale, and 2) we could convert millions of devices into stress sensors with virtually no added cost. We plan to extend our method to study other body parts and emotions. On the road we could detect neck stiffness from head oscillations using cameras or leg stiffness by comparing an IMU attached to the chassis and a pressure sensor located on the surface of the seat.
Auto-nomic: Turning the car into an affective sensor.
PeriStress: Peripherals and furniture as stress sensors.
Mind Mining: Mining stressors from social media.
We are working on the development of a new science of subtle interventions “in the wild”, i.e., using everyday devices and activities to deliver just-in-time relief. We have harvested popular apps, games, and media that “intersect” with therapeutic techniques, and recommend the right intervention for the right situation using online learning algorithms. As wemove to physical devices, we have started to explore three non-traditional interactions: a) non-volitional, “force” adoption of healthy behaviors, e.g., an opt-out robotic sit-stand-desk that changes height periodically; b) subliminal perception, changing biorhythms without the user’s awareness, e.g., imperceptible hue oscillations from a display to excite the visual cortex; and c) interstitials, repurpose in-between times, e.g., using the commute to decompress. Across the board, we expect higher engagement via multisensorial stimulation, e.g. combining haptics and sound; and discovering effective popular interventions from large-scale social media databases.
False Biofeedback: Providing fake biofeedback to users to promote stress relief.
Self-Standing Desk: A desk to promote healthy office habits.
Light Therapy: A room saturated with different light hues to promote stress relief.
Breathing Browser: A Chrome extension to make web browsing more pleasant.
Oscillating Chair: A chair that provides oscillations based on gait analysis.
Subliminal Reality: Imperceptible modification of biorhythms.
Given that stressed people report having no time to cope with it, we are fascinated with in-between moments, such as the commute, as an opportunity to break the stress “chain” between work and home. We have created sensors that reuse the angular displacement of a steering wheel and built an interactive car seat that uses haptic actuators to guide drivers of manual or autonomous vehicles to move their body or slow down their breathing. Given the safety risks of driving under high stress, we follow a parsimonious approach: we start with Wizard-of-Oz exploration and controlled experiments in a driving simulator, validate them with closed-circuit studies, to finally move to on-road experiments. As we design for fully autonomous cars, we are the first to have shown evidence that a well-designed VR experience can help relax its users. The private nature of the commute, makes it an idea scenario to explore therapeutic chatbots. As we master the car, we plan to explore other modes of transportation, and extend my prior research on interactive affective urban lights that increase walkability. Our work is sponsored by the Center for Automotive Research as Stanford (CARS), and car companies such as Renault, Faurecia and Toyota.
Fast and Furious 2: Detecting stress on the road.
Hapty Seat: Breathing & movement interventions for commuters.
Cool ReMix: Calming & mindful mixed reality in the car.
Fast & Furious: Detecting stress with a steering wheel.
Fiat Lux: Affective urban lights to promote walking.
CYBER-PHYSICAL CONVERSATIONAL THERAPEUTIC AGENTS
As we advance the design of interactive systems, it is natural to assume a convergence with conversational agents. Our vision is to have an ecosystem of cyber-physical therapeutic agents to help people deal proactively with daily hassles. Based on my work on just-in-time interventions we have built an “army of micro chatbots”, which are deployed to provide a single stress "tip,” such as somatic relaxation, cognitive reframing, etc. We have early evidence that people value variety and expect these bots to co-exist with human advisors. We have built a text version for the phone and a voice version for the car. As we collect data we plan to use HCI, DSP and AI techniques to solve hard problems such as parsing a stressor from a compound statement, crafting personalized empathic affirmations, and managing end-point and turn-taking in voice-only chatbots, e.g. detecting speech pauses with prosody and natural-language processing, while designing interactions such as “are you done?” or “hum!”. As we move forward, our flexible voice platform (a.k.a. “Carla”) allows us to test novel ideas such as a non-antropomorphic "dog bot," that converses with empathic pet sounds, or a “car bot” that gives a massage with the seat after a word of advice.
PopBots: an "army" of tiny chatbots for stress management.
PopTherapy: Repurposing popular media into coping strategies.
DogBots: Automated dog audio responses for stress management.