Advances in large language models (LLMs) have empowered a variety of applications. However, there is still a significant gap in research when it comes to understanding and enhancing the capabilities of LLMs in the field of mental health. In this work, we present a comprehensive evaluation of multiple LLMs on various mental health prediction tasks via online text data, including Alpaca, Alpaca-LoRA, FLAN-T5, GPT-3.5, and GPT-4. We conduct a broad range of experiments, covering zero-shot prompting, few-shot prompting, and instruction fine-tuning. The results indicate a promising yet limited performance of LLMs with zero-shot and few-shot prompt designs for mental health tasks. More importantly, our experiments show that instruction finetuning can significantly boost the performance of LLMs for all tasks simultaneously. Our best-finetuned models, Mental-Alpaca and Mental-FLAN-T5, outperform the best prompt design of GPT-3.5 (25 and 15 times bigger) by 10.9% on balanced accuracy and the best of GPT-4 (250 and 150 times bigger) by 4.8%. They further perform on par with the state-of-the-art task-specific language model. We also conduct an exploratory case study on LLMs' capability on mental health reasoning tasks, illustrating the promising capability of certain models such as GPT-4. We summarize our findings into a set of action guidelines for potential methods to enhance LLMs' capability for mental health tasks. Meanwhile, we also emphasize the important limitations before achieving deployability in real-world mental health settings, such as known racial and gender bias. We highlight the important ethical risks accompanying this line of research.

A deep understanding of how discrimination impacts psychological health and well-being of students could allow us to better protect individuals at risk and support those who encounter discrimination. While the link between discrimination and diminished psychological and physical well-being is well established, existing research largely focuses on chronic discrimination and long-term outcomes. A better understanding of the short-term behavioral correlates of discrimination events could help us to concretely quantify such experiences, which in turn could support policy and intervention design. In this paper we specifically examine, for the first time, what behaviors change and in what ways in relation to discrimination. We use actively-reported and passively-measured markers of health and well-being in a sample of 209 first-year college students over the course of two academic quarters. We examine changes in indicators of psychological state in relation to reports of unfair treatment in terms of five categories of behaviors: physical activity, phone usage, social interaction, mobility, and sleep. We find that students who encounter unfair treatment become more physically active, interact more with their phone in the morning, make more calls in the evening, and spend more time in bed on the day of the event. Some of these patterns continue the next day. Our results further our understanding of the impact of discrimination and can inform intervention work.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Loneliness significantly affects the quality of life and physical and mental health. In addition, recent studies have shown high levels of loneliness across various populations ranging from older adults to college students. Detection of loneliness through passive sensing on personal devices can lead to a better understanding of measurable behaviors that low and high loneliness individuals exhibit and can thus lead to explore these behaviors in interventions aimed at decreasing rates of loneliness among college students. Objective: We aimed to explore the potential of using smartphone and wearable sensors to infer the severity of loneliness and to identify its corresponding behavioral patterns. Methods: Data was collected from smartphones and Fitbits of 160 college students over a semester. Participants completed the UCLA loneliness questionnaire at the beginning and at the end of the semester. The scores were categorized to high (questionnaire score>40) and low (<=40) loneliness. Daily features were extracted from mobile phone and Fitbit sensors to capture activity and mobility, communication and phone usage, and sleep behaviors. The features were then aggregated to generate semester-level features. We used three analytic methods: 1) statistical analysis to provide an overview of loneliness in college students, 2) data mining using the Apriori algorithm to extract combined behavior patterns associated with loneliness, and 3) machine learning classification using an ensemble of Gradient Boosting and Logistic Regression algorithms with feature selection in a leave-one-student-out cross-validation manner to infer the level of loneliness. Results: The average loneliness score from both the pre-surveys and post-surveys was high, above 43, and the majority of participants fell into the high loneliness category, with 63.8% in pre-survey and 58.5% in post-survey. Severe loneliness (scores greater than one standard deviation above the mean) increased from 38.2% at the beginning of the semester to 39.4% at the end of the semester. Our machine learning pipeline achieved an accuracy of 80.2% in detecting the binary level of loneliness and an 88.4% accuracy in detecting change in loneliness level. The mining of associations between classifier-selected behavioral features and loneliness indicated that (1) spending less time outside of campus during evening hours on weekends and (2) spending less time in Greek houses for social events in the evening and night on weekdays were common behavior patterns in students with high loneliness. The analysis also indicated more activity and less sedentary behavior especially in the evening and night was associated with a decrease in loneliness. Conclusions: Passive sensing has the potential for detecting loneliness in college students and identifying the associated daily behavioral patterns, including the intensity of activity especially during night hours and the amount of time spent outside of campus. These findings highlight opportunities for interventions through mobile technology to reduce the impact of loneliness and social isolation on individuals’ health and wellbeing.

We conducted a questionnaire-based study of the relative importance of two factors, inquirer and situation, in determining the preferred accuracy of personal information disclosed through a ubiquitous computing system. We found that privacy preferences varied by inquirer more than by situation. That is, individuals were more likely to apply the same privacy preferences to the same inquirer in different situations than to apply the same privacy preferences to different inquirers in the same situation. We are applying these results to the design of a user interface for managing everyday privacy in ubiquitous computing.

There is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM- short for Generalization of Longitudinal BEhavior Modeling - to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers' attention to model generalizability evaluation for future longitudinal human behavior modeling studies.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet