This study develops a comprehensive robotic system, termed the robot cognitive system, for complex environments, integrating three models: the engagement model, the intention model, and the human–robot interaction (HRI) model. The system aims to enhance the naturalness and comfort of HRI by enabling robots to detect human behaviors, intentions, and emotions accurately. A novel dual-arm-hand mobile robot, Mobi, was designed to demonstrate the system’s efficacy. The engagement model utilizes eye gaze, head pose, and action recognition to determine the suitable moment for interaction initiation, addressing potential eye contact anxiety. The intention model employs sentiment analysis and emotion classification to infer the interactor’s intentions. The HRI model, integrated with Google Dialogflow, facilitates appropriate robot responses based on user feedback. The system’s performance was validated in a retail environment scenario, demonstrating its potential to improve the user experience in HRIs.

With the spread of generative AI, the study proposed a memory-based cognitive robot architecture by using a Large Language Model (LLM), inspired by the working memory of the human brain model. A card-pairing task is designed to perform visual working memory with 60 participants with the measurement of electroencephalography (EEG). The proposed human brain model is a muti-featured EEG model, represented by the concept of brain energy, connectivity, and complexity. Band power ratios show that the anti-correlation of alpha and gamma waveforms can be observed in the occipital lobe. Brain connectivity is represented through magnitude squared coherence and phase locking value, and brain complexity is calculated by the Katz fractal dimension. Three pathways—attention, short-term memory, and distraction resistance are revealed. The changes in Katz fractal dimension are discovered in the frontal and occipital lobes. A novel memory architecture of robot cognition designed from human brain model includes three types of memory—short-term memory, working memory, and long-term memory, which is driven by GPT-4o. Two memory tests, a recalling test, and a working memory test, are conducted to validate the memory ability of the robot. The precision / recall / F1-score of working memory performance is 1.0 / 0.35 / 0.519, and the average accuracy of recalling test is 0.7. It is demonstrated that the different types of human memory functions can be implemented in the cognitive robot architecture driven by LLM. The research not only provides insight into the working memory of the human brain model but also realizes the robot architecture with the application of generative AI.

1. Consider the opening minutes of a social robot’s first conversation with an older adult. Within seconds, the older adult immediately begins forming impressions: is this robot warm or cold, emoti...