Frank Joublin

Frank Joublin

Auto-generated profile, based on publicly available information

Frank Joublin

Honda Research Institute Europe

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
2 May 2024
Verified
Conference Paper
Open access
CWChao Wang
Chao Wang
SHStephan Hasler
Stephan Hasler
DTDaniel Tanneberg
Daniel Tanneberg
FOFelix Ocker
Felix Ocker
FJFrank Joublin
Frank Joublin
Michael GiengerMichael Gienger
Michael Gienger
This paper presents an innovative large language model (LLM)-based robotic system for enhancing multi-modal human-robot interaction (HRI). Traditional HRI systems relied on complex designs for intent estimation, reasoning, and behavior generation, which were resource-intensive. In contrast, our system empowers researchers and practitioners to regulate robot behavior through three key aspects: providing high-level linguistic guidance, creating "atomic actions" and expressions the robot can use, and offering a set of examples. Implemented on a physical robot, it demonstrates proficiency in adapting to multi-modal inputs and determining the appropriate manner of action to assist humans with its arms, following researchers’ defined guidelines. Simultaneously, it coordinates the robot’s lid, neck, and ear movements with speech output to produce dynamic, multi-modal expressions. This showcases the system’s potential to revolutionize HRI by shifting from conventional, manual state-and-flow design methods to an intuitive, guidance-based, and example-driven approach. Supplementary material can be found at https://hri-eu.github.io/Lami/

21 August 2025
Verified
Book Chapter
Closed access
PDF file
ASAhmed Sadik
Ahmed Sadik
SBSebastian Brulin
Sebastian Brulin
MOMarkus Olhofer
Markus Olhofer
ACAntonello Ceravola
Antonello Ceravola
FJFrank Joublin
Frank Joublin

1 March 2008 in International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
Verified
Conference Paper
Closed access
HBHolger Brandl
Holger Brandl
FJFrank Joublin
Frank Joublin
CGChristian Goerick
Christian Goerick
Understanding the bootstrapping process of speech representation in infants is one key issue towards systems which may provide humanlike speech recognition abilities some day. Until now, almost all current speech recognition systems have failed to integrate learning into the recognition process. Here we propose a system for unsupervised word-clustering, which is able to recognize and learn the structure of speech online in a unified framework. To do so we've extended HMM-based filler-free keyword spotting with acoustic model acquisition. To evaluate and control the dynamics of the combined acquisition-recognition process we propose measures for model activity, model correlation and speech coverage.

30 May 2025
Verified
Book Chapter
Closed access
PDF file
EIEbubechukwu Ike
Ebubechukwu Ike
JTJohane Takeuchi
Johane Takeuchi
FJFrank Joublin
Frank Joublin
ACAntonello Ceravola
Antonello Ceravola
MTMarc Tanti
Marc Tanti

15 February 2020 in arXiv
Verified
Preprint
Open access
PDF file
CWChao Wang
Chao Wang
SHStephan Hasler
Stephan Hasler
MMManuel Muehlig
Manuel Muehlig
FJFrank Joublin
Frank Joublin
ACAntonello Ceravola
Antonello Ceravola
LFLydia Fischer
Lydia Fischer
Future intelligent system will involve very various types of artificial agents, such as mobile robots, smart home infrastructure or personal devices, which share data and collaborate with each other to execute certain tasks.Designing an efficient human-machine interface, which can support users to express needs to the system, supervise the collaboration progress of different entities and evaluate the result, will be challengeable. This paper presents the design and implementation of the human-machine interface of Intelligent Cyber-Physical system (ICPS),which is a multi-entity coordination system of robots and other smart devices in a working environment. ICPS gathers sensory data from entities and then receives users' command, then optimizes plans to utilize the capability of different entities to serve people. Using multi-model interaction methods, e.g. graphical interfaces, speech interaction, gestures and facial expressions, ICPS is able to receive inputs from users through different entities, keep users aware of the progress and accomplish the task efficiently

11 May 2026 in ACM Transactions on Interactive Intelligent Systems
Verified
Journal Article
Open access
JGJoy Geuenich
Joy Geuenich
FJFrank Joublin
Frank Joublin
ACAntonello Ceravola
Antonello Ceravola
SBStefan Brandenburg
Stefan Brandenburg
JDJonathan Dodge
Jonathan Dodge
Inclusive usability testing, such as the GenderMag method, wants to identify gender-related usability problems in digital interfaces. Large Language Models (LLMs) have been used by usability engineers in usability evaluations but their contribution is still underexplored, especially regarding inclusive usability testing. Research has shown that GenderMag workshops can produce valuable insights but are resource intense and might show effects from the evaluators’ ability to embody personas with a different cognitive style. Therefore, we need to assess if LLM-agent based testing can aid human-based evaluations. This study evaluates an LLM-agent system for GenderMag persona-based usability testing, and compares its performance to traditional human-led evaluations. The agent system integrates GenderMag persona facets into three LLM-agents, which analyze usability issues of four web interfaces, three generic and one intentionally flawed interface with gender-related usability issues. We quantitatively and qualitatively compare the types, severity, and relevance of usability problems LLM-agents identified to those produced in three GenderMag workshops involving nine participants. Findings show a broad overlap in detecting usability issues of humans and LLM-agents that are not gender specific via the generic interfaces. Agents thereby consistently assign significantly higher severity and relevance ratings. For the intentionally flawed interface, humans and LLM-agents assign similar ratings but the overlap between human and agent gender-related usability issues was low, with each missing issues the other caught. The agent system is an efficient tool for gender-related usability evaluation that humans may overlook, thereby expanding the coverage of evaluation.