Jonathan Fields

Jonathan Fields

Auto-generated profile, based on publicly available information

Jonathan Fields

Knight Foundation School of Computing and Information Sciences, Florida International University, USA

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 November 2024
Verified
Conference Paper
Open access
JMJoaquin Molto
Joaquin Molto
JFJonathan Fields
Jonathan Fields
UVUbbo Visser
Ubbo Visser
CLChristine Lisetti
Christine Lisetti
Virtual Socially Interactive Agents (SIA) have shown great promise for human interactions with computer applications in which not only domain-relevant content is needed, but also the way in which the content is delivered (e.g. socio-emotionally adaptive tutoring agents, socio-emotionally responsive health agents). While recent progress on Large Language Models (LLMs) has made rich verbal interactions possible, LLMs cannot communicate nonverbal social cues through a simple text-based interface. We propose an expressive conversational SIA system, powered by an OpenAI Large Language Model (LLM) for text generation, integrated with a 3D humanoid model with real-time behavior generation of FACS-based facial expressions that mirror the user’s to increase rapport and engagement using HumeAI’s Facial Expression Recognition and Empathic Voice Interface (EVI) models to drive the model’s animations. As a case study, we use prompt-engineering to focus the conversation on discussing health-related behaviors. We ground the generation of the LLM’s questions based on the World Health Organization’s (WHO) Alcohol Use Disorders Identification Test (AUDIT) 10-question inventory, a test that help identify whether someone is at risk of alcohol use disorder.