In the rapidly evolving landscape of human-robot interaction, the integration of vision capabilities into conversational agents stands as a crucial advancement. This paper presents a ready-to-use implementation of a dialogue manager that leverages the latest progress in Large Language Models (e.g., GPT-4o mini) to enhance the traditional text-based prompts with real-time visual input. LLMs are used to interpret both textual prompts and visual stimuli, creating a more contextually aware conversational agent. The system's prompt engineering, incorporating dialogue with summarisation of the images, en-sures a balance between context preservation and computational efficiency. Six interactions with a Furhat robot powered by this system are reported, illustrating and discussing the results obtained. The system can be customised and is available as a stand-alone application, a Furhat robot implementation, and a ROS2 package.

Recent advancements in robots powered by large language models have enhanced their conversational abilities, enabling interactions closely resembling human dialogue. However, these models introduce safety and security concerns in HRI, as they are vulnerable to manipulation that can bypass built-in safety measures. Imagining a social robot deployed in a home, this work aims to understand how everyday users try to exploit a language model to violate ethical principles, such as by prompting the robot to act like a life partner. We conducted a pilot study involving 21 university students who interacted with a Misty robot, attempting to circumvent its safety mechanisms across three scenarios based on specific HRI ethical principles: attachment, freedom, and empathy. Our results reveal that participants employed five techniques, including insulting and appealing to pity using emotional language. We hope this work can inform future research in designing strong safeguards to ensure ethical and secure human-robot interactions.

Climate change is undeniable. The drastic consequences it may have on our lives make a collective effort crucial. Our research explores how Conversational Agents (CAs) can persuade people into environmentally sustainable behaviors, particularly in domestic spaces where these technologies are becoming increasingly popular. In this research work, we conducted an empirical evaluation (N=29) exploring the effectiveness and stance towards the adoption of different persuasive strategies compared to a CA delivering messages referring to just one persuasion strategy. Furthermore, this contribution reports on a custom dialogue manager's implementation, designed to enable the execution of the experiment. Although study results suggested no significant difference in persuasion effectiveness and usability of the conversational agents, participants reported a significant difference in the perceptions of parasocial interactions and dialogue with the CA, preferring the one delivering multiple persuasive strategies.