Deep Learning for Deep Breathing

A perception project aimed at developing new engagement and skill-building features for the Ommie robot. A custom dataset was collected of individuals performing deep breathing with a respiration belt and a number of robot-based sensors. Current work uses this dataset to apply machine learning techniques (feed-forward, convolutional, and recurrent) to predict deep breathing phases (inhales, hold, exhales).

Full paper published in the ICMI '23: Proceedings of the 25th International Conference on Multimodal Interaction: https://doi.org/10.1145/3577190.3614173

Abstract:
Social robots are in a unique position to aid mental health by supporting engagement with behavioral interventions. One such behavioral intervention is the practice of deep breathing, which has been shown to physiologically reduce symptoms of anxiety. Multiple robots have been recently developed that support deep breathing, but none yet implement a method to detect how accurately an individual is performing the practice. Detecting breathing phases (i.e., inhaling, breath holding, or exhaling) is a challenge with these robots since often the robot is being manipulated or moved by the user, or the robot itself is moving to generate haptic feedback. Accordingly, we first present OMMDB: a novel, multimodal, public dataset made up of individuals performing deep breathing with an Ommie robot in multiple conditions of robot ego-motion. The dataset includes RGB video, inertial sensor data, and motor encoder data, as well as ground truth breathing data from a respiration belt. Our second contribution features experimental results with a convolutional long-short term memory neural network trained using OMMDB. These results show the system’s ability to be applied to the domain of deep breathing and generalize between individual users. We additionally show that our model is able to generalize across multiple types of robot ego-motion, reducing the need to train individual models for varying human-robot interaction conditions.