How are you Feeling? Determining the Latent State of Individuals Engaged in Conversation
Inferring the latent emotive content of a narrative requires consideration of para-linguistic cues, linguistic content and the physiological state of the narrator. In this study we utilized a combination of auditory, text, and physiological signals to predict the mood (happy/sad) of 31 narrations from subjects engaged in personal story-telling. We extracted 500+ audio and physiological features from the data, and explored the effects of introducing our selected features at various layers of the Neural Network. We evaluated our model’s performance using leave-one-subject-out cross-validation and compared it to 20 baseline models. To ensure the real-time utility of the model, classification was performed over 5 second intervals.
Tuka Alhanai is a PhD candidate in Computer Science at MIT, where she focuses on the development of machine learning algorithms in the context of speech and language processing for health applications. Tuka’s current work leverages multi-modal data to develop automated tools that assess an individual's emotional and mental well-being, such as depression and dementia, and is currently collaborating with the Framingham Heart Study in this line of research. Tuka was awarded an MIT Legatum Fellowship for pursuing entrepreneurial ventures in the developing world, with her work being featured in Wired and Newsweek.