Park CW, et al. J Korean Med Sci. 2020 Oct;35(42):e379. https://doi.org/10.3346/jkms.2020.35.e379

Background: Federated learning (FL) is a newly proposed machine-learning method that uses a decentralized dataset. Since data transfer is not necessary for the learning process in FL, there is a significant advantage in protecting personal privacy. Therefore, many studies are being actively conducted in the applications of FL for diverse areas. Objective: The aim of this study was to evaluate the reliability and performance of FL using three benchmark datasets, including a clinical benchmark dataset. Methods: To evaluate FL in a realistic setting, we implemented FL using a client-server architecture with Python. The implemented client-server version of the FL software was deployed to Amazon Web Services. Modified National Institute of Standards and Technology (MNIST), Medical Information Mart for Intensive Care-III (MIMIC-III), and electrocardiogram (ECG) datasets were used to evaluate the performance of FL. To test FL in a realistic setting, the MNIST dataset was split into 10 different clients, with one digit for each client. In addition, we conducted four different experiments according to basic, imbalanced, skewed, and a combination of imbalanced and skewed data distributions. We also compared the performance of FL to that of the state-of-the-art method with respect to in-hospital mortality using the MIMIC-III dataset. Likewise, we conducted experiments comparing basic and imbalanced data distributions using MIMIC-III and ECG data. Results: FL on the basic MNIST dataset with 10 clients achieved an area under the receiver operating characteristic curve (AUROC) of 0.997 and an F1-score of 0.946. The experiment with the imbalanced MNIST dataset achieved an AUROC of 0.995 and an F1-score of 0.921. The experiment with the skewed MNIST dataset achieved an AUROC of 0.992 and an F1-score of 0.905. Finally, the combined imbalanced and skewed experiment achieved an AUROC of 0.990 and an F1-score of 0.891. The basic FL on in-hospital mortality using MIMIC-III data achieved an AUROC of 0.850 and an F1-score of 0.944, while the experiment with the imbalanced MIMIC-III dataset achieved an AUROC of 0.850 and an F1-score of 0.943. For ECG classification, the basic FL achieved an AUROC of 0.938 and an F1-score of 0.807, and the imbalanced ECG dataset achieved an AUROC of 0.943 and an F1-score of 0.807. Conclusions: FL demonstrated comparative performance on different benchmark datasets. In addition, FL demonstrated reliable performance in cases where the distribution was imbalanced, skewed, and extreme, reflecting the real-life scenario in which data distributions from various hospitals are different. FL can achieve high performance while maintaining privacy protection because there is no requirement to centralize the data.

Despite the growing adoption of the mobile health (mHealth) applications (apps), few studies address concerns with low retention rates. This study aimed to investigate how the usage patterns of mHealth app functions affect user retention. We collected individual usage logs for 1,439 users of single tethered personal health record app, which spanned an 18-months period from August 2011 to January 2013. The user logs contained timestamps whenever an individual uses each function, which enables us to identify the usage patterns based on the intensity of using a particular function in the app. We then estimated how these patterns were related to 1) the app usage over time (using the random effect model) and 2) the probability of stopping the use of the application (using the Cox proportional hazard model). The analyses suggested that the users utilize the app most at the time of the adoption and gradually reduce their usage over time. The average duration of use after starting the app was 25.62 weeks (SD: 18.41). The degree of the usage reduction, however, decreases as the self-monitoring function is more frequently used (coefficient = 0.002, P = 0.013); none of the other functions has this effect. Moreover, engaging with the self-monitoring function frequently (coefficient = −0.18, P = 0.003) and regularly (coefficient = 0.10, P = 0.001) significantly also reduces the probability of abandoning the application. Specifically, the estimated survival rate indicates that, after 40 weeks since the adoption, the probability of the regular users of self-monitoring to stay in use was about 80% while that of non-user was about 60%. This study provides the empirical evidence that sustained use of mHealth app is closely linked to the regular usage on self-monitoring function. The implications can be extended to the education of users and physicians to produce better outcomes as well as application development for effective user interfaces.

Background: Mobile mental-health trackers are mobile phone apps that gather self-reported mental-health ratings from users. They have received great attention from clinicians as tools to screen for depression in individual patients. While several apps that ask simple questions using face emoticons have been developed, there has been no study examining the validity of their screening performance. Objective: In this study, we (1) evaluate the potential of a mobile mental-health tracker that uses three daily mental-health ratings (sleep satisfaction, mood, and anxiety) as indicators for depression, (2) discuss three approaches to data processing (ratio, average, and frequency) for generating indicator variables, and (3) examine the impact of adherence on reporting using a mobile mental-health tracker and accuracy in depression screening. Methods: We analyzed 5792 sets of daily mental-health ratings collected from 78 breast cancer patients over a 48-week period. Using the Patient Health Questionnaire-9 (PHQ-9) as the measure of true depression status, we conducted a random-effect logistic panel regression and receiver operating characteristic (ROC) analysis to evaluate the screening performance of the mobile mental-health tracker. In addition, we classified patients into two subgroups based on their adherence level (higher adherence and lower adherence) using a k-means clustering algorithm and compared the screening accuracy between the two groups. Results: With the ratio approach, the area under the ROC curve (AUC) is 0.8012, indicating that the performance of depression screening using daily mental-health ratings gathered via mobile mental-health trackers is comparable to the results of PHQ-9 tests. Also, the AUC is significantly higher (P=.002) for the higher adherence group (AUC=0.8524) than for the lower adherence group (AUC=0.7234). This result shows that adherence to self-reporting is associated with a higher accuracy of depression screening. Conclusions: Our results support the potential of a mobile mental-health tracker as a tool for screening for depression in practice. Also, this study provides clinicians with a guideline for generating indicator variables from daily mental-health ratings. Furthermore, our results provide empirical evidence for the critical role of adherence to self-reporting, which represents crucial information for both doctors and patients.

Shin SY, et al. J Korean Med Sci. 2015 Jan;30(1):7-15. https://doi.org/10.3346/jkms.2015.30.1.7

Shin SY. Korean J Physiol Pharmacol. 2019 Sep;23(5):311-315. https://doi.org/10.4196/kjpp.2019.23.5.311