3 September 2025 in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Emotional and mental well-being are vital components of quality of life, and with the rise of smart devices like smartphones, wearables, and artificial intelligence (AI), new opportunities for monitoring emotions in everyday settings have emerged. However, for AI algorithms to be effective, they require high-quality data and accurate annotations. As the focus shifts towards collecting emotion data in real-world environments to capture more authentic emotional experiences, the process of gathering emotion annotations has become increasingly complex. This work explores the challenges of everyday emotion data collection from the perspectives of key stakeholders. We collected 75 survey responses, performed 32 interviews with the public, and 3 focus group discussions (FGDs) with 12 mental health professionals. The insights gathered from a total of 119 stakeholders informed the development of our framework, AnnoSense, which comprises 15 actionable guidelines designed to support everyday emotion data collection for AI systems. This framework was then evaluated by 25 emotion AI experts for its clarity, usefulness, and adaptability. Lastly, we discuss the potential next steps and implications of AnnoSense for future research in emotion AI, highlighting its potential to enhance the collection and analysis of emotion data in real-world contexts.
2 May 2025 in Proceedings of the ACM on Human-Computer Interaction
Physiological signals hold immense potential for ubiquitous emotion monitoring, presenting numerous applications in emotion recognition. However, harnessing this potential is hindered by significant challenges, particularly in the collection of annotations that align with physiological changes since the process hinges heavily on human participants. In this work, we set out to study human participants' perspectives in the emotion data collection procedure. We conducted a lab-based emotion data collection study with 37 participants using 360° virtual reality video stimulus followed by semi-structured interviews with the study participants. Our findings presented that intrinsic factors like participants' perception, experiment design nuances, and experiment setup suitability impact their emotional response and annotation within lab settings. Drawing from our findings and prior research, we propose recommendations for incorporating participants' context into annotations and emphasizing participant-centric experiment designs. Furthermore, we explore current emotion data collection practices followed by AI practitioners and offer insights for future contributions leveraging physiological emotion data.