“Imagine if, at the end of a conversation, you could rewind it and see the moments when the people around you felt the most anxious,” Tuka Alhanai, a CSAIL graduate student who worked on the project, said in a statement. “Our work is a step in this direction, suggesting that we may not be that far away from a world where people can have an AI social coach right in their pocket.”
Using two AI algorithms and a Samsung Simband, the system measures vitals like heart rate, blood pressure, blood flow, and skin temperature, while recording audio data related to the wearer’s pitch, tone, and vocabulary. It then attempts to identify the speaker’s mood, from positive to neutral to negative, in five-second increments.
“As far as we know, this is the first experiment that collects both physical data and speech data in a passive but robust way, even while subjects are having natural, unstructured interactions,” Mohammad Ghassemi, a CSAIL doctoral student who worked on the project, said in a statement. “Our results show that it’s possible to classify the emotional tone of conversations in real time.”
The system is imperfect and not yet ready for real-world applications, but it’s better than current methods. Results showed it could identify moods with an accuracy of about 18 percent above chance and 7.5 percent better than existing technologies. Alhanai and Ghassemi will present their findings at Association for the Advancement of Artificial Intelligence (AAAI) conference in San Francisco next week.
