The idea of introducing a robot into an Ambient Assisted Living (AAL) environment to provide additional services beyond those provided by the environment itself has been explored in numerous projects. Moreover, new opportunities can arise from this symbiosis, which usually requires both systems to share the knowledge (and not just the data) they capture from the context. Thus, by using knowledge extracted from the raw data captured by the sensors deployed in the environment, the robot can know where the person is and whether he/she should perform some physical exercise, as well as whether he/she should move a chair away to allow the robot to successfully complete a task. This paper describes the design of an Ambient Assisted Living system where an IoT scheme and robot coexist as independent but connected elements, forming a cyber-physical system-of-systems architecture. The IoT environment includes cameras to monitor the person’s activity and physical position (lying down, sitting…), as well as non-invasive sensors to monitor the person’s heart or breathing rate while lying in bed or sitting in the living room. Although this manuscript focuses on how both systems handle and share the knowledge they possess about the context, a couple of example use cases are included. In the first case, the environment provides the robot with information about the positions of objects in the environment, which allows the robot to augment the metric map it uses to navigate, detecting situations that prevent it from moving to a target. If there is a person nearby, the robot will approach them to ask them to move a chair or open a door. In the second case, even more use is made of the robot’s ability to interact with the person. When the IoT system detects that the person has fallen to the ground, it passes this information to the robot so that it can go to the person, talk to them, and ask for external help if necessary.

In recent years, there has been growing interest in developing robots capable of explaining their behavior, thereby improving their acceptance by humans with whom they share their environment. Proposed software designs are typically based on the advances being made in conversational systems built on deep learning techniques. However, apart from the ability to formulate explanations, the robot also needs an internal episodic memory, where it stores information from the continuous stream of experiences. Most previous proposals are designed to deal with short streams of episodic data (several minutes long). With the aim of managing larger experiences, we propose in this work a high-level episodic memory, where relevant events are abstracted to natural language concepts. The proposed framework is intimately linked to a software architecture in which the explanations, whether externalized or not, are shaped internally in a collaborative process involving the task-oriented software agents that make up the architecture. The core of this process is a runtime knowledge model, employed as working memory whose evolution allows for capturing the causal events stored in the episodic memory. We present several use cases that illustrate how the suggested framework allows an autonomous robot to generate correct and relevant explanations of its actions and behavior.

Ambient Assisted Living (AAL) environments aim to enable older people to remain active and lead an autonomous and independent life for as long as possible. Among the technologies that can be incorporated into these settings, socially assistive robots (SAR) seek to establish a more natural and intuitive means of interaction with people, whilst helping them to carry out everyday tasks. One of the main challenges facing the design of these robots is how to enable them to undertake more complex tasks. Recent advances in Large Language Models (LLMs) have opened new avenues for flexible robot deliberation, yet their integration into real-time robotic systems remains challenging due to latency constraints, reasoning reliability, and the complexity of coordinating multi-step tasks. This paper proposes a hierarchical multi-agent architecture for robot deliberation that addresses these challenges by combining LLM-based planning with structured execution mechanisms within the ROS 2 ecosystem. The proposed architecture employs a supervisor agent that decomposes high-level natural language instructions into prioritised subtasks, enabling a priority-driven execution model that dynamically adapts to task relevance, temporal constraints, and environmental feedback. Subtasks are delegated to a set of Single-Purpose Agents (SPAs), orchestrated via LangGraph state machines and coordinated through a priority-aware scheduling mechanism. A key design principle is the use of Behaviour Trees (BTs) as high-level callable tools through the Model Context Protocol (MCP), encapsulating closed-loop control strategies while enabling preemptive and priority-consistent execution. This reduces the number of LLM inference steps required per task and improves robustness under dynamic conditions. A further contribution concerns the deployment of fine-tuned, lightweight LLMs—on the order of 0.6 billion parameters—specifically adapted for both the supervisor and the individual SPA roles through parameter-efficient low-rank adaptation (LoRA). These models are trained on role-specific tool-calling datasets to specialise in constrained reasoning patterns and task-specific decision-making, enabling efficient, low-latency inference directly on edge hardware. The combination of fine-tuning and hierarchical priority control enhances both the determinism and responsiveness of the system while mitigating error propagation across agent interactions. The paper presents the full software architecture, a formal characterisation of the system as a priority-aware hierarchical policy over a graph of agent workflows, and an experimental evaluation in an Ambient Assisted Living scenario assessing task success rate, inference efficiency, responsiveness under competing priorities, and overall user experience. Because SPA execution is decoupled from the supervisor’s own reasoning loop, the architecture is designed to keep accepting, processing, and queuing new user queries while previously dispatched SPAs are still executing their tasks.