Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Worldwide, the burden of chronic diseases is growing, necessitating novel approaches that complement and go beyond evidence-based medicine. In this respect a promising avenue is the secondary use of Electronic Health Records (EHR) data, where clinical data are analysed to conduct basic and clinical and translational research. Methods based on machine learning algorithms to process EHR are resulting in improved understanding of patients’ clinical trajectories and chronic disease risk prediction, creating a unique opportunity to derive previously unknown clinical insights. However, wealth of patients’ clinical history remains locked behind clinical narratives in free-form text. Consequently, unlocking the full potential of EHR data is contingent on development of Natural Language Processing (NLP) methods to automatically transform clinical text into structured clinical data that can be directly processed using machine learning algorithms. Objective: To provide a comprehensive overview of the development and uptake on NLP methods applied to free-text clinical notes related to chronic diseases, including investigation of challenges faced by NLP methodologies in understanding clinical narratives. Methods: The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed and searches were conducted in 5 databases using “clinical notes”, “natural language processing” and “chronic disease” as keywords as well as their variations to maximise coverage of the articles. Results: Of the 2646 articles considered, 100 met the inclusion criteria. Review of the included papers resulted in identification of 42 chronic diseases, which were then further classified into 10 diseases categories using ICD-10. Majority of the studies focused on diseases of circulatory system (N=38) while endocrine and metabolic diseases were fewest (N=12). This was due to the structure of clinical records related to metabolic diseases that typically contain much more structured data than medical records for diseases of circulatory system, which focus more on unstructured data and consequently have seen a stronger focus of NLP. The review has shown that there is a significant increase in the use of machine learning methods compared to rule-based approaches, however deep learning methods remain emergent (N=3). Consequently, majority of works focus on classification of disease phenotype, while only a handful of papers concern the extraction of comorbidities from the free-text or the integration of clinical notes with structured data. There is a notable use of relatively simple methods, such as shallow classifiers (or combination with rule-based methods), due to the interpretability of predictions, which still represents a significant issue for more complex methods. Finally, scarcity of publicly available data may also have contributed to insufficient development of more advanced methods, such as extraction of word embeddings from clinical notes. Conclusions: Efforts are needed to improve (1) progression of clinical NLP methods from extraction towards understanding; (2) recognition of relations among entities, rather than entities in isolation; (3) temporal extraction to understand past, current and future clinical events; (4) exploitation of alternative sources of clinical knowledge; and (5) availability of large-scale, de-identified clinical corpora.

Optimal performance is desired for decision-making in any field with binary classifiers and diagnostic tests, however common performance measures lack depth in information. The area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve are too general because they evaluate all decision thresholds including unrealistic ones. Conversely, accuracy, sensitivity, specificity, positive predictive value and the F1 score are too specific—they are measured at a single threshold that is optimal for some instances, but not others, which is not equitable. In between both approaches, we propose deep ROC analysis to measure performance in multiple groups of predicted risk (like calibration), or groups of true positive rate or false positive rate. In each group, we measure the group AUC (properly), normalized group AUC, and averages of: sensitivity, specificity, positive and negative predictive value, and likelihood ratio positive and negative. The measurements can be compared between groups, to whole measures, to point measures and between models. We also provide a new interpretation of AUC in whole or part, as balanced average accuracy, relevant to individuals instead of pairs. We evaluate models in three case studies using our method and Python toolkit and confirm its utility.

Today's health care is difficult to imagine without the possibility to objectively measure various physiological parameters related to patients' symptoms (from temperature through blood pressure to complex tomographic procedures). Psychiatric care remains a notable exception that heavily relies on patient interviews and self-assessment. This is due to the fact that mental illnesses manifest themselves mainly in the way patients behave throughout their daily life and, until recently there were no “behavior measurement devices.” This is now changing with the progress in wearable activity recognition and sensor enabled smartphones. In this paper, we introduce a system, which, based on smartphone-sensing is able to recognize depressive and manic states and detect state changes of patients suffering from bipolar disorder. Drawing upon a real-life dataset of ten patients, recorded over a time period of 12 weeks (in total over 800 days of data tracing 17 state changes) by four different sensing modalities, we could extract features corresponding to all disease-relevant aspects in behavior. Using these features, we gain recognition accuracies of 76% by fusing all sensor modalities and state change detection precision and recall of over 97%. This paper furthermore outlines the applicability of this system in the physician-patient relations in order to facilitate the life and treatment of bipolar patients.

Increase in workload across many organizations and consequent increase in occupational stress are negatively affecting the health of the workforce. Measuring stress and other human psychological dynamics is difficult due to subjective nature of selfreporting and variability between and within individuals. With the advent of smartphones, it is now possible to monitor diverse aspects of human behavior, including objectively measured behavior related to psychological state and consequently stress. We have used data from the smartphone's built-in accelerometer to detect behavior that correlates with subjects stress levels. Accelerometer sensor was chosen because it raises fewer privacy concerns (e.g., in comparison to location, video, or audio recording), and because its low-power consumption makes it suitable to be embedded in smaller wearable devices, such as fitness trackers. About 30 subjects from two different organizations were provided with smartphones. The study lasted for eight weeks and was conducted in real working environments, with no constraints whatsoever placed upon smartphone usage. The subjects reported their perceived stress levels three times during their working hours. Using combination of statistical models to classify selfreported stress levels, we achieved a maximum overall accuracy of 71% for user-specific models and an accuracy of 60% for the use of similar-users models, relying solely on data from a single accelerometer.