Yahui Fu

Yahui Fu

Auto-generated profile, based on publicly available information

Yahui Fu

Graduate School of Informatics, Kyoto University, Kyoto, Japan

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 May 2022 in IEEE MultiMedia
Verified
Journal Article
Closed access
YFYahui Fu
Yahui Fu
Shogo OkadaShogo Okada
Shogo Okada
LWLongbiao Wang
Longbiao Wang
Lili GuoLili Guo
Lili Guo
YSYaodong Song
Yaodong Song
JDJianwu Dang
Jianwu Dang
This work proposes an approach for emotion recognition in conversation that leverages context modeling, knowledge enrichment, and multimodal (text and audio) learning based on a graph convolutional network (GCN). We first construct two distinctive graphs for modeling the contextual interaction and knowledge dynamic. We then introduce an affective lexicon into knowledge graph building to enrich the emotional polarity of each concept, that is the related knowledge of each token in an utterance. Then, we achieve a balance between the context and the affect-enriched knowledge by incorporating them into the new adjacency matrix construction of the GCN architecture, and teach them jointly with multiple modalities to effectively structure the semantics-sensitive and knowledge-sensitive contextual dependence of each conversation. Our model outperforms the state-of-the-art benchmarks by over 22.6% and 11% relative error reduction in terms of weighted-F1 on the IEMOCAP and MELD databases, respectively, demonstrating the superiority of our method in emotion recognition.

23 March 2022 in IEEE MultiMedia
Verified
Journal Article
Closed access
Lili GuoLili Guo
Lili Guo
LWLongbiao Wang
Longbiao Wang
JDJianwu Dang
Jianwu Dang
YFYahui Fu
Yahui Fu
JLJiaxing Liu
Jiaxing Liu
SDShifei Ding
Shifei Ding
People usually express emotions through paralinguistic and linguistic information in speech. How to effectively integrate linguistic and paralinguistic information for emotion recognition is a challenge. Previous studies have adopted the bidirectional long short-term memory (BLSTM) network to extract acoustic and lexical representations followed by a concatenate layer, and this has become a common method. However, the interaction and influence between different modalities are difficult to promote using simple feature fusion for each sentence. In this article, we propose an implicitly aligned multimodal transformer fusion (IA-MMTF) framework based on acoustic features and text information. This model enables the two modalities to guide and complement each other when learning emotional representations. Thereafter, the weighed fusion is used to control the contributions of different modalities. Thus, we can obtain more complementary emotional representations. Experiments on the interactive emotional dyadic motion capture (IEMOCAP) database and multimodal emotionlines dataset (MELD) show that the proposed method outperforms the baseline BLSTM-based method.

20 October 2023 in Advanced Robotics
Verified
Journal Article
Closed access
YFYahui Fu
Yahui Fu
KIKoji Inoue
Koji Inoue
DLDivesh Lala
Divesh Lala
KYKenta Yamamoto
Kenta Yamamoto
CCChenhui Chu
Chenhui Chu
TKTatsuya Kawahara
Tatsuya Kawahara