Jieyeon Woo

Jieyeon Woo

Auto-generated profile, based on publicly available information

Jieyeon Woo

ISIR, Sorbonne University, France

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 October 2023
Verified
Conference Paper
Open access
TKTaras Kucherenko
Taras Kucherenko
RNRajmund Nagy
Rajmund Nagy
YYYoungwoo Yoon
Youngwoo Yoon
JWJieyeon Woo
Jieyeon Woo
TNTeodor Nikolov
Teodor Nikolov
GHGustav Eje Henter
Gustav Eje Henter
This paper reports on the GENEA Challenge 2023, in which participating teams built speech-driven gesture-generation systems using the same speech and motion dataset, followed by a joint evaluation. This year’s challenge provided data on both sides of a dyadic interaction, allowing teams to generate full-body motion for an agent given its speech (text and audio) and the speech and motion of the interlocutor. We evaluated 12 submissions and 2 baselines together with held-out motion-capture data in several large-scale user studies. The studies focused on three aspects: 1) the human-likeness of the motion, 2) the appropriateness of the motion for the agent’s own speech whilst controlling for the human-likeness of the motion, and 3) the appropriateness of the motion for the behaviour of the interlocutor in the interaction, using a setup that controls for both the human-likeness of the motion and the agent’s own speech. We found a large span in human-likeness between challenge submissions, with a few systems rated close to human mocap. Appropriateness seems far from being solved, with most submissions performing in a narrow range slightly above chance, far behind natural motion. The effect of the interlocutor is even more subtle, with submitted systems at best performing barely above chance. Interestingly, a dyadic system being highly appropriate for agent speech does not necessarily imply high appropriateness for the interlocutor. Additional material is available via the project website at svito-zar.github.io/GENEAchallenge2023/.

19 June 2024 in International Journal of Human-Computer Studies
Verified
Journal Article
Open access
JWJieyeon Woo
Jieyeon Woo
KSKazuhiro Shidara
Kazuhiro Shidara
CACatherine Achard
Catherine Achard
HTHiroki Tanaka
Hiroki Tanaka
SNSatoshi Nakamura
Satoshi Nakamura
CPCatherine Pelachaud
Catherine Pelachaud

27 March 2023
Verified
Conference Paper
Open access
JWJieyeon Woo
Jieyeon Woo
CPCatherine Pelachaud
Catherine Pelachaud
CACatherine Achard
Catherine Achard
Socially Interactive Agents (SIAs) offer users with interactive face-to-face conversations. They can take the role of a speaker and communicate verbally and nonverbally their intentions and emotional states; but they should also act as active listener and be an interactive partner. In human-human interaction, interlocutors adapt their behaviors reciprocally and dynamically. The endowment of such adaptation capability can allow SIAs to show social and engaging behaviors. In this paper, we focus on modelizing the reciprocal adaptation to generate SIA behaviors for both conversational roles of speaker and listener. We propose the Augmented Self-Attention Pruning (ASAP) neural network model. ASAP incorporates recurrent neural network, attention mechanism of transformers, and pruning technique to learn the reciprocal adaptation via multimodal social signals. We evaluate our work objectively, via several metrics, and subjectively, through a user perception study where the SIA behaviors generated by ASAP is compared with those of other state-of-the-art models. Our results demonstrate that ASAP significantly outperforms the state-of-the-art models and thus shows the importance of reciprocal adaptation modeling.

19 September 2023
Verified
Conference Paper
Open access
JWJieyeon Woo
Jieyeon Woo
MGMichele Grimaldi
Michele Grimaldi
CPCatherine Pelachaud
Catherine Pelachaud
CACatherine Achard
Catherine Achard
During an interaction, partners adapt their behaviors to each other. Adaptation can have several functions such as being a sign of engagement and enhancing human users' interaction experience. It is important that virtual agents acting as interaction partners should continuously adapt their behaviors to those of their interlocutors in real time. This paper focuses on creating an interactive virtual agent that is capable of rendering real-time adaptive behaviors in response to its human interlocutor. It ensures the two aspects: generating real-time adaptive behavior and managing natural dialogue. We propose a system of an adaptive virtual agent and choose the e-health application of Cognitive Behavioral Therapy (CBT), which is a mental health treatment that restructures automatic thoughts into balanced thoughts, as a proof-of-concept to showcase the benefit of endowing behavior adaptation to the agent. The virtual agent adapts to the user via the display of nonverbal behaviors, which are generated via a deep learning model, throughout the whole interaction while acting as a therapist helping human users to detect their negative automatic thoughts.

9 October 2023
Verified
Conference Paper
Open access
TSTakeshi Saga
Takeshi Saga
JWJieyeon Woo
Jieyeon Woo
AGAlexis Gerard
Alexis Gerard
HTHiroki Tanaka
Hiroki Tanaka
CACatherine Achard
Catherine Achard
CPCatherine Pelachaud
Catherine Pelachaud
Interlocutors adapt their verbal and nonverbal behaviors as signs of engagement during face-to-face interaction. We aim to build engaging Socially Interactive Agents, SIAs, that can adapt their behaviors during interaction. With an adaptive behavior generation model, we drive SIAs’ upper face and head movements in real-time. We evaluate this platform through a scenario for Social Skills Training, SST.

9 July 2023
Verified
Book Chapter
Closed access
PDF file
JWJieyeon Woo
Jieyeon Woo
LYLiu Yang
Liu Yang
CPCatherine Pelachaud
Catherine Pelachaud
CACatherine Achard
Catherine Achard