Shivam Mehta

Shivam Mehta

Auto-generated profile, based on publicly available information

Shivam Mehta

Division of Speech, Music and Hearing (TMH), KTH Royal Institute of Technology, Sweden

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 October 2023
Verified
Conference Paper
Open access
ADAnna Deichler
Anna Deichler
SMShivam Mehta
Shivam Mehta
SASimon Alexanderson
Simon Alexanderson
Jonas BeskowJonas Beskow
Jonas Beskow
This paper describes a system developed for the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. Our solution builds on an existing diffusion-based motion synthesis model. We propose a contrastive speech and motion pretraining (CSMP) module, which learns a joint embedding for speech and gesture with the aim to learn a semantic coupling between these modalities. The output of the CSMP module is used as a conditioning signal in the diffusion-based gesture synthesis model in order to achieve semantically-aware co-speech gesture generation. Our entry achieved highest human-likeness and highest speech appropriateness rating among the submitted entries. This indicates that our system is a promising approach to achieve human-like co-speech gestures in agents that carry semantic meaning.