The decisions made by autonomous robots hold substantial influence over how humans perceive their behavior. One way to alleviate potential negative impressions of such decisions by humans and enhance human comprehension of them is through explaining. We introduce visual and textual explanations integrated into robot navigation, considering the surrounding environmental context. To gauge the effectiveness of our approach, we conducted a comprehensive user study, assessing user satisfaction across different forms of explanation representation. Our empirical findings reveal a notable discrepancy in user satisfaction, with significantly higher levels observed for explanations that adopt a multimodal format, as opposed to those relying solely on unimodal representations.

Manipulating MW in driving simulator studies without the need to introduce a non-driving-related task remains challenging. This study aims to empirically evaluate the modified speed regulation n-back task, a tool to manipulate drivers’ MW. Our experiment involved 23 participants who experienced a 0-back and 2-back driving condition, with task-irrelevant novel environmental sounds used to elicit P3a event-related potentials. Results indicate that the 2-back condition was perceived as more demanding, evidenced by higher NASA-TLX scores (overall score, mental and temporal demand, effort, frustration). The mean P3a amplitude was diminished during the 2-back condition compared to the 0-back condition, suggesting that drivers experienced higher MW and had fewer resources available to process the novel environmental sounds. This study provides empirical evidence indicating that the speed regulation n-back task could be a valid, effective, and reproducible method to manipulate MW in driving research.

In recent years we have seen large language models (LLMs) demonstrating robust reasoning capabilities comparable to human performance. This makes them increasingly appealing for driver assistance, where adaptation to dynamic human context is essential. Yet, research in this area remains fragmented, often focusing on isolated applications, lacking utilization of LLM’s full potential to deliver integrated, context-specific support and action. This survey synthesizes recent advancements in LLM-driven occupant monitoring systems, focusing on their capabilities for interpreting driver states and acting appropriately, enabling a new generation of intelligent driver assistance. We critically examine pioneering frameworks, benchmarks, and foundational datasets that employ techniques like reasoning chains, multimodality, and human-in-the-loop feedback to create personalized and safe driving experiences. We lay out the current trends, limitations, and emerging patterns, in addition to a novel human-centered evaluation of the field, providing researchers with a roadmap towards transparent and trustworthy in-cabin systems that bridge safety with driver experience.

Driver emotions such as stress, frustration, and anxiety are associated with riskier behavior and reduced situational awareness, motivating in-vehicle systems that regulate affect in real time. However, emotion-only approaches capture how a driver feels, but not why, limiting appropriate support delivery. We present Emotune, a work-in-progress framework integrating emotional state assessment, driving-context awareness, and Large Language Model (LLM)-assisted intervention selection within the CARLA simulator. The framework uses a human-guided workflow, in which predefined interventions are selected by locally deployed LLM agents. As a methodological contribution, it enables studying how contextual information can inform emotion regulation in driving. We report a pilot study comparing a baseline emotion-aware agent with a context-aware agent that grounds interventions in driving situations. No statistically significant between-condition differences were found in the pilot study. Qualitative feedback instead highlighted contextual relevance, intervention timing, and perceived empathy as priorities for future investigation.

In recent years we have seen Large Language Models (LLMs) demonstrating robust reasoning capabilities comparable to human performance. This makes them increasingly appealing for driver assistance, where adaptation to dynamic human context is essential. Yet, research in this area remains fragmented, often focusing on isolated applications, lacking utilization of LLM's full potential to deliver integrated, context-specific support and action. This survey synthesizes recent advancements in LLM-driven occupant monitoring systems, focusing on their capabilities for interpreting driver states and acting appropriately, enabling a new generation of intelligent driver assistance. We critically examine pioneering frameworks, benchmarks, and foundational datasets that employ techniques like reasoning chains, multimodality, and human-in-the-loop feedback to create personalized and safe driving experiences. We lay out the current trends, limitations, emerging patterns, in addition to a novel human-centered evaluation of the field, providing researchers with a roadmap towards transparent and trustworthy in-cabin systems, that bridge safety with driver experience.