Masafumi Nishimura

Prof. Dr. Masafumi Nishimura

Graduate School of Science and Technology, Shizuoka University
orcid

Research Focus & Metrics

Academic Career Length

16Years (2011-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 May 2021 in Sensors
Verified
Journal Article
Open access
PDF file
ANAkihiro Nakamura
Akihiro Nakamura
TSTakato Saito
Takato Saito
DIDaizo Ikeda
Daizo Ikeda
KOKen Ohta
Ken Ohta
HMHiroshi Mineno
Hiroshi Mineno
MNMasafumi Nishimura
Masafumi Nishimura
A series of eating behaviors, including chewing and swallowing, is considered to be crucial to the maintenance of good health. However, most such behaviors occur within the human body, and highly invasive methods such as X-rays and fiberscopes must be utilized to collect accurate behavioral data. A simpler method of measurement is needed in healthcare and medical fields; hence, the present study concerns the development of a method to automatically recognize a series of eating behaviors from the sounds produced during eating. The automatic detection of left chewing, right chewing, front biting, and swallowing was tested through the deployment of the hybrid CTC/attention model, which uses sound recorded through 2ch microphones under the ear and weak labeled data as training data to detect the balance of chewing and swallowing. N-gram based data augmentation was first performed using weak labeled data to generate many weak labeled eating sounds to augment the training data. The detection performance was improved through the use of the hybrid CTC/attention model, which can learn the context. In addition, the study confirmed a similar detection performance for open and closed foods.

13 January 2023 in JMIR Formative Research
Verified
Journal Article
Open access
KYKenta Yoshii
Kenta Yoshii
DKDaiki Kimura
Daiki Kimura
AKAkihiro Kosugi
Akihiro Kosugi
KSKaoru Shinkawa
Kaoru Shinkawa
TTToshiro Takase
Toshiro Takase
MNMasafumi Nishimura
Masafumi Nishimura
Background: The rising number of patients with dementia has become a serious social problem worldwide. To help detect dementia at an early stage, many studies have been conducted to detect signs of cognitive decline by prosodic and acoustic features. However, many of these methods are not suitable for everyday use as they focus on cognitive function or conversational speech during the examinations. In contrast, conversational humanoid robots are expected to be used in the care of older people to help reduce the work of care and monitoring through interaction. Objective: This study focuses on early detection of mild cognitive impairment (MCI) through conversations between patients and humanoid robots without a specific examination, such as neuropsychological examination. Methods: This was an exploratory study involving patients with MCI and cognitively normal (CN) older people. We collected the conversation data during neuropsychological examination (Mini-Mental State Examination [MMSE]) and everyday conversation between a humanoid robot and 94 participants (n=47, 50%, patients with MCI and n=47, 50%, CN older people). We extracted 17 types of prosodic and acoustic features, such as the duration of response time and jitter, from these conversations. We conducted a statistical significance test for each feature to clarify the speech features that are useful when classifying people into CN people and patients with MCI. Furthermore, we conducted an automatic classification experiment using a support vector machine (SVM) to verify whether it is possible to automatically classify these 2 groups by the features identified in the statistical significance test. Results: We obtained significant differences in 5 (29%) of 17 types of features obtained from the MMSE conversational speech. The duration of response time, the duration of silent periods, and the proportion of silent periods showed a significant difference (P<.001) and met the reference value r=0.1 (small) of the effect size. Additionally, filler periods (P<.01) and the proportion of fillers (P=.02) showed a significant difference; however, these did not meet the reference value of the effect size. In contrast, we obtained significant differences in 16 (94%) of 17 types of features obtained from the everyday conversations with the humanoid robot. The duration of response time, the duration of speech periods, jitter (local, relative average perturbation [rap], 5-point period perturbation quotient [ppq5], difference of difference of periods [ddp]), shimmer (local, amplitude perturbation quotient [apq]3, apq5, apq11, average absolute differences between the amplitudes of consecutive periods [dda]), and F0cov (coefficient of variation of the fundamental frequency) showed a significant difference (P<.001). In addition, the duration of response time, the duration of silent periods, the filler period, and the proportion of fillers showed significant differences (P<.05). However, only jitter (local) met the reference value r=0.1 (small) of the effect size. In the automatic classification experiment for the classification of participants into CN and MCI groups, the results showed 66.0% accuracy in the MMSE conversational speech and 68.1% accuracy in everyday conversations with the humanoid robot. Conclusions: This study shows the possibility of early and simple screening for patients with MCI using prosodic and acoustic features from everyday conversations with a humanoid robot with the same level of accuracy as the MMSE.

1 April 2018 in Speech Communication
Verified
Journal Article
Closed access
TFTakashi Fukuda
Takashi Fukuda
OIOsamu Ichikawa
Osamu Ichikawa
MNMasafumi Nishimura
Masafumi Nishimura

9 March 2021
Verified
Conference Paper
Closed access
KYKenta Yoshii
Kenta Yoshii
MNMasafumi Nishimura
Masafumi Nishimura
DKDaiki Kimura
Daiki Kimura
AKAkihiro Kosugi
Akihiro Kosugi
KSKaoru Shinkawa
Kaoru Shinkawa
SHShinji Higashi
Shinji Higashi
Many studies have attempted to detect dementia in patients early by identifying certain prosodic and acoustic features during neuropsychological examinations. Conversational humanoid robots are expected to be used in elderly care to help increase their quality of life through interactions. If we can detect cognitive impairments from the elderly's conversations with a robot, it would aid in early detection of dementia. In this study, we collected conversational data to simulate daily conversations with humanoid robots and compared them between healthy older adults and patients with mild cognitive impairments. This study evaluated multiple speech features and observed significant differences in answer length, response length, duration of utterance, jitter, and shimmer. Thus, the study results suggest the possibility of early detection of dementia through conversations with a humanoid robot.

5 June 2024 in Applied Sciences
Verified
Journal Article
Open access
PDF file
MMMaliha Binte Mamun
Maliha Binte Mamun
TTTakashi Tsunakawa
Takashi Tsunakawa
MNMasafumi Nishida
Masafumi Nishida
MNMasafumi Nishimura
Masafumi Nishimura
The growing number of social media users has impacted the rise in hate comments and posts. While extensive research in hate speech detection attempts to combat this phenomenon by developing new datasets and detection models, reconciling classification accuracy with broader decision-making metrics like plausibility and faithfulness remains challenging. As restrictions on social media tighten to stop the spread of hate and offensive content, users have adapted by finding new approaches, often camouflaged in the form of sarcasm. Therefore, dealing with new trends such as the increased use of emoticons (negative emoticons in positive sentences) and sarcastic comments is necessary. This paper introduces sarcasm-based rationale (emoticons or portions of text that indicate sarcasm) combined with hate/offensive rationale for better detection of hidden hate comments/posts. A dataset was created by labeling texts and selecting rationale based on sarcasm from the existing benchmark hate dataset, HateXplain. The newly formed dataset was then applied in the existing state-of-the-art model. The model’s F1-score increased by 0.01 when using sarcasm rationale with hate/offensive rationale in a newly formed attention proposed in the data’s preprocessing step. Also, with the new data, a significant improvement was observed in explainability metrics such as plausibility and faithfulness.

27 August 2011
Verified
Conference Paper
Closed access
MSMasayuki Suzuki
Masayuki Suzuki
GKGakuto Kurata
Gakuto Kurata
MNMasafumi Nishimura
Masafumi Nishimura
NMNobuaki Minematsu
Nobuaki Minematsu