Shun Katada

Shun Katada

Auto-generated profile, based on publicly available information

Shun Katada

Japan Advanced Institute of Science and Technology, Japan

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 March 2022 in IEEE Transactions on Affective Computing
Verified
Journal Article
Closed access
SKShun Katada
Shun Katada
Shogo OkadaShogo Okada
Shogo Okada
KKKazunori Komatani
Kazunori Komatani
Multimodal sentiment analysis has become a focus of research in recent years. However, most studies of multimodal sentiment analysis have considered only signals that are observable by humans, such as linguistic, audio and visual information, whereas the contribution of the multimodal fusion of such signals with unobservable signals, i.e., physiological signals, has not been comprehensively explored. In this study, we aim to investigate effects of physiological signals in multimodal sentiment analysis by evaluating all of the fusion models for different types of sentiment estimation in naturalistic human-agent interaction settings. Our results suggest that physiological features are effective in the unimodal model and that the fusion of linguistic representations with physiological features provides the best results for estimating self-sentiment labels as annotated by the users themselves. In contrast, the tensor fusion of linguistic representations with audiovisual features is effective for estimating sentiment labels as annotated by a third party in regression tasks, which can be derived from the corresponding signals that are observable by humans. A detailed analysis of the self-sentiment estimation results suggests that different modalities play different roles in sentiment estimation, and corresponding implications are discussed.

5 April 2022 in Multimedia Tools and Applications
Verified
Journal Article
Closed access
PDF file
SKShun Katada
Shun Katada
Shogo OkadaShogo Okada
Shogo Okada

9 June 2023 in JMIR Research Protocols
Verified
Journal Article
Open access
MHMasanobu Hibi
Masanobu Hibi
SKShun Katada
Shun Katada
AKAya Kawakami
Aya Kawakami
KBKotatsu Bito
Kotatsu Bito
MOMayumi Ohtsuka
Mayumi Ohtsuka
HMHiroshi Maruyama
Hiroshi Maruyama
Background: Human health status can be measured on the basis of many different parameters. Statistical relationships among these different health parameters will enable several possible health care applications and an approximation of the current health status of individuals, which will allow for more personalized and preventive health care by informing the potential risks and developing personalized interventions. Furthermore, a better understanding of the modifiable risk factors related to lifestyle, diet, and physical activity will facilitate the design of optimal treatment approaches for individuals. Objective: This study aims to provide a high-dimensional, cross-sectional data set of comprehensive health care information to construct a combined statistical model as a single joint probability distribution and enable further studies on individual relationships among the multidimensional data obtained. Methods: In this cross-sectional observational study, data were collected from a population of 1000 adult men and women (aged ≥20 years) matching the age ratio of the typical adult Japanese population. Data include biochemical and metabolic profiles from blood, urine, saliva, and oral glucose tolerance tests; bacterial profiles from feces, facial skin, scalp skin, and saliva; messenger RNA, proteome, and metabolite analyses of facial and scalp skin surface lipids; lifestyle surveys and questionnaires; physical, motor, cognitive, and vascular function analyses; alopecia analysis; and comprehensive analyses of body odor components. Statistical analyses will be performed in 2 modes: one to train a joint probability distribution by combining a commercially available health care data set containing large amounts of relatively low-dimensional data with the cross-sectional data set described in this paper and another to individually investigate the relationships among the variables obtained in this study. Results: Recruitment for this study started in October 2021 and ended in February 2022, with a total of 997 participants enrolled. The collected data will be used to build a joint probability distribution called a Virtual Human Generative Model. Both the model and the collected data are expected to provide information on the relationships between various health statuses. Conclusions: As different degrees of health status correlations are expected to differentially affect individual health status, this study will contribute to the development of empirically justified interventions based on the population. International Registered Report Identifier (IRRID): DERR1-10.2196/47024

7 November 2022
Verified
Conference Paper
Open access
SKShun Katada
Shun Katada
Shogo OkadaShogo Okada
Shogo Okada
KKKazunori Komatani
Kazunori Komatani
One of the main challenges in realizing dialog systems is adapting to a user’s sentiment state in real time. Large-scale language models, such as BERT, have achieved excellent performance in sentiment estimation; however, the use of only linguistic information from user utterances in sentiment estimation still has limitations. In fact, self-reported sentiment is not necessarily expressed by user utterances. To mitigate the issue that the true sentiment state is not expressed as observable signals, psychophysiology and affective computing studies have focused on physiological signals that capture involuntary changes related to emotions. We address this problem by efficiently introducing time-series physiological signals into a state-of-the-art language model to develop an adaptive dialog system. Compared with linguistic models based on BERT representations, physiological long short-term memory (LSTM) models based on our proposed physiological signal processing method have competitive performance. Moreover, we extend our physiological signal processing method to the Transformer language model and propose the Time-series Physiological Transformer (TPTr), which captures sentiment changes based on both linguistic and physiological information. In ensemble models, our proposed methods significantly outperform the previous best result (p < 0.05).

27 November 2022
Verified
Conference Paper
Open access
TOTomoya Ohba
Tomoya Ohba
CMCandy Olivia Mawalim
Candy Olivia Mawalim
SKShun Katada
Shun Katada
HKHaruki Kuroki
Haruki Kuroki
Shogo OkadaShogo Okada
Shogo Okada
An interview for a job recruiting process requires applicants to demonstrate their communication skills. Interviewees sometimes become nervous about the interview because interviewees themselves do not know their assessed score. This study investigates the relationship between the communication skill (CS) and the self-efficacy level (SE) of interviewees through multimodal modeling. We also clarify the difference between effective features in the prediction of CS and SE labels. For this purpose, we collect a novel multimodal job interview data corpus by using a job interview agent system where users experience the interview using a virtual reality head-mounted display (VR-HMD). The data corpus includes annotations of CS by third-party experts and SE annotations by the interviewees. The data corpus also includes various kinds of multimodal data, including audio, biological (i.e., physiological), gaze, and language data. We present two types of regression models, linear regression and sequential-based regression models, to predict CS, SE, and the gap (GA) between skill and self-efficacy. Finally, we report that the model with acoustic, gaze, and linguistic features has the best regression accuracy in CS prediction (correlation coefficient r = 0.637). Furthermore, the regression model with biological features achieves the best accuracy in SE prediction (r = 0.330).

25 January 2025 in medRxiv
Verified
Preprint
Open access
PDF file
HMHiroshi Maruyama
Hiroshi Maruyama
KBKotatsu Bito
Kotatsu Bito
YSYuki Saito
Yuki Saito
MHMasanobu Hibi
Masanobu Hibi
SKShun Katada
Shun Katada
SMShin-Ichi Maeda
Shin-Ichi Maeda
Data for healthcare applications are typically customized for specific purposes but are often difficult to access due to high costs and privacy concerns. Rather than prepare separate datasets for individual applications, we propose a novel approach: building a general-purpose generative model applicable to virtually any type of healthcare application. This generative model encompasses a broad range of human attributes, including age, sex, anthropometric measurements, blood components, physical performance metrics, and numerous healthcare-related questionnaire responses. To achieve this goal, we integrated the results of multiple clinical studies into a unified training dataset and developed a generative model to replicate its characteristics. The model can estimate missing attribute values from known attribute values and generate synthetic datasets for various applications. Our analysis confirmed that the model captures key statistical properties of the training dataset, including univariate distributions and bivariate relationships. We demonstrate the model’s practical utility through multiple real-world applications, illustrating its potential impact on predictive, preventive, and personalized medicine.