Background: We developed a system for monitoring mental health using voice data from daily phone calls, termed Mind Monitoring System (MIMOSYS), by implementing a method for estimating mental health status from voice data.Objective: The objective of this study was to evaluate the potential of this system for detecting depressive states and monitoring stress-induced mental changes.Methods: We opened our system to the public in the form of a prospective study in which data were collected over 2 years from a large, unspecified sample of users. We used these data to analyze the relationships between the rate of continued use, the men-to-women ratio, and existing psychological tests for this system over the study duration. Moreover, we analyzed changes in mental data over time under stress from particular life events.Results: The system had a high rate of continued use. Voice indicators showed that women have more depressive tendencies than men, matching the rate of depression in Japan. The system’s voice indicators and the scores on classical psychological tests were correlated. We confirmed deteriorating mental health for users in areas affected by major earthquakes in Japan around the time of the earthquakes.Conclusions: The results suggest that although this system is insufficient for detecting depression, it may be effective for monitoring changes in mental health due to stress. The greatest feature of our system is mental health monitoring, which is most effectively accomplished by performing long-term time-series analysis of the acquired data considering the user’s life events. Such a system can improve the implementation of patient interventions by evaluating objective data along with life events.

Background: In many developed countries, mood disorders have become problematic, and the economic loss due to treatment costs and interference with work is immeasurable. Therefore, a simple technique to determine individuals’ depressive state and stress level is desired. Methods: We developed a method to assess specific the psychological issues of individuals with major depressive disorders using emotional components contained in their voice. We propose two indices: vitality, a short-term index, and mental activity, a long-term index capturing trends in vitality. To evaluate our method, we used the voices of healthy individuals (n = 14) and patients with major depression (n = 30). The patients were also assessed by specialists using the Hamilton Rating Scale for Depression (HAM-D). Results: A significant negative correlation existed between the vitality extracted from the voices and HAM-D scores (r = −0.33, p < 0.05). Furthermore, we could discriminate the voice data of healthy individuals and patients with depression with a high accuracy using the vitality indicator (p = 0.0085, area under the curve of the receiver operating characteristic curve = 0.76).

Recently, the relationship between emotional arousal and depression has been studied. Focusing on this relationship, we first developed an arousal level voice index (ALVI) to measure arousal levels using the Interactive Emotional Dyadic Motion Capture database. Then, we calculated ALVI from the voices of depressed patients from two hospitals (Ginza Taimei Clinic (H1) and National Defense Medical College hospital (H2)) and compared them with the severity of depression as measured by the Hamilton Rating Scale for Depression (HAM-D). Depending on the HAM-D score, the datasets were classified into a no depression (HAM-D < 8) and a depression group (HAM-D ≥ 8) for each hospital. A comparison of the mean ALVI between the groups was performed using the Wilcoxon rank-sum test and a significant difference at the level of 10% (p = 0.094) at H1 and 1% (p = 0.0038) at H2 was determined. The area under the curve (AUC) of the receiver operating characteristic was 0.66 when categorizing between the two groups for H1, and the AUC for H2 was 0.70. The relationship between arousal level and depression severity was indirectly suggested via the ALVI.

Autism spectrum disorder (ASD) is often signaled by atypical cries during infancy. Copy number variants (CNVs) provide genetically identifiable cases of ASD, but how early atypical cries predict a later onset of ASD among CNV carriers is not understood in humans. Genetic mouse models of CNVs have provided a reliable tool to experimentally isolate the impact of CNVs and identify early predictors for later abnormalities in behaviors relevant to ASD. However, many technical issues have confounded the phenotypic characterization of such mouse models, including systematically biased genetic backgrounds and weak or absent behavioral phenotypes. To address these issues, we developed a coisogenic mouse model of human proximal 16p11.2 hemizygous deletion and applied computational approaches to identify hidden variables within neonatal vocalizations that have predictive power for postpubertal dimensions relevant to ASD. After variables of neonatal vocalizations were selected by least absolute shrinkage and selection operator (Lasso), random forest, and Markov model, regression models were constructed to predict postpubertal dimensions relevant to ASD. While the average scores of many standard behavioral assays designed to model dimensions did not differentiate a model of 16p11.2 hemizygous deletion and wild-type littermates, specific call types and call sequences of neonatal vocalizations predicted individual variability of postpubertal reciprocal social interaction and olfactory responses to a social cue in a genotype-specific manner. Deep-phenotyping and computational analyses identified hidden variables within neonatal social communication that are predictive of postpubertal behaviors.

In general, it is common knowledge that people’s feelings are reflected in their voice and facial expressions. This research work focuses on developing techniques for diagnosing depression based on acoustic properties of the voice. In this study, we developed a composite index of vocal acoustic properties that can be used for depression detection. Voice recordings were collected from patients undergoing outpatient treatment for major depressive disorder at a hospital or clinic following a physician’s diagnosis. Numerous features were extracted from the collected audio data using openSMILE software. Furthermore, qualitatively similar features were combined using principal component analysis. The resulting components were incorporated as parameters in a logistic regression based classifier, which achieved a diagnostic accuracy of ~90% on the training set and ~80% on the test set. Lastly, the proposed metric could serve as a new measure for evaluation of major depressive disorder.