Youssef Nassar

Youssef Nassar

Auto-generated profile, based on publicly available information

Youssef Nassar

University Reutlingen Alteburgstraße 150 72762 Reutlingen Germany

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 March 2025 in IEEE Robotics & Automation Magazine
Verified
Journal Article
Closed access
YLYuzhi Lai
Yuzhi Lai
Shenghai YuanShenghai Yuan
Shenghai Yuan
YNYoussef Nassar
Youssef Nassar
MFMingyu Fan
Mingyu Fan
AGAtmaraaj Gopal
Atmaraaj Gopal
MRMatthias Rätsch
Matthias Rätsch
Translating human intent into robot commands is crucial for the future of service robots in an aging society. Existing human‒robot interaction (HRI) systems relying on gestures or verbal commands are impractical for the elderly, due to difficulties with complex syntax or sign language. To address the challenge, this article introduces a multimodal interaction framework that combines voice and deictic posture information to create a more natural HRI system. Visual cues are first processed by the object detection model to gain a global understanding of the environment, and then bounding boxes are estimated based on depth information. By using a large language model (LLM) with voice-to-text commands and temporally aligned selected bounding boxes, robot action sequences can be generated, while key control syntax constraints are applied to avoid potential LLM hallucination issues. The system is evaluated on real-world tasks with varying levels of complexity, using a Universal Robots UR3e manipulator. Our method demonstrates significantly better HRI performance in terms of accuracy and robustness. To benefit the research community and the general public, we made our code and design open source.

5 January 2025 in Expert Systems with Applications
Verified
Journal Article
Closed access
YLYuzhi Lai
Yuzhi Lai
Shenghai YuanShenghai Yuan
Shenghai Yuan
YNYoussef Nassar
Youssef Nassar
MFMingyu Fan
Mingyu Fan
TWThomas Weber
Thomas Weber
MRMatthias Rätsch
Matthias Rätsch

8 May 2024
Verified
Book Chapter
Closed access
PDF file
YLYuzhi Lai
Yuzhi Lai
MRMario Radke
Mario Radke
YNYoussef Nassar
Youssef Nassar
AGAtmaraaj Gopal
Atmaraaj Gopal
TWThomas Weber
Thomas Weber
MRMatthias Rätsch
Matthias Rätsch

8 May 2024
Verified
Book Chapter
Closed access
PDF file
YNYoussef Nassar
Youssef Nassar
MRMario Radke
Mario Radke
AGAtmaraaj Gopal
Atmaraaj Gopal
TKTobias Knöller
Tobias Knöller
TWThomas Weber
Thomas Weber
MRMatthias Rätsch
Matthias Rätsch

12 March 2025 in arXiv
Verified
Preprint
Open access
PDF file
YLYuzhi Lai
Yuzhi Lai
Shenghai YuanShenghai Yuan
Shenghai Yuan
YNYoussef Nassar
Youssef Nassar
MFMingyu Fan
Mingyu Fan
TWThomas Weber
Thomas Weber
MRMatthias Rätsch
Matthias Rätsch
Effective Human-Robot Interaction (HRI) is crucial for future service robots in aging societies. Existing solutions are biased toward only well-trained objects, creating a gap when dealing with new objects. Currently, HRI systems using predefined gestures or language tokens for pretrained objects pose challenges for all individuals, especially elderly ones. These challenges include difficulties in recalling commands, memorizing hand gestures, and learning new names. This paper introduces NVP-HRI, an intuitive multi-modal HRI paradigm that combines voice commands and deictic posture. NVP-HRI utilizes the Segment Anything Model (SAM) to analyze visual cues and depth data, enabling precise structural object representation. Through a pre-trained SAM network, NVP-HRI allows interaction with new objects via zero-shot prediction, even without prior knowledge. NVP-HRI also integrates with a large language model (LLM) for multimodal commands, coordinating them with object selection and scene distribution in real time for collision-free trajectory solutions. We also regulate the action sequence with the essential control syntax to reduce LLM hallucination risks. The evaluation of diverse real-world tasks using a Universal Robot showcased up to 59.2\% efficiency improvement over traditional gesture control, as illustrated in the video https://youtu.be/EbC7al2wiAc. Our code and design will be openly available at https://github.com/laiyuzhi/NVP-HRI.git.