Background: During the COVID-19 pandemic, the general public was concerned about the mental health impacts of unemployment due to COVID-19 and the stress essential workers experienced during this time. Several reports indicated that people in distress were turning to digital technology, but there was little evidence about the impact of these tools on mitigating distress. Objective: This study seeks to determine the acceptability, feasibility, usability, and effectiveness of mobile mental health apps for decreasing mental health symptoms in essential workers and unemployed individuals with suicide risk. Methods: We recruited participants who indicated that they were unemployed because of COVID-19 or were COVID-19–designated essential workers. Participants were randomized to 1 of 4 free commercial mobile apps for managing distress that were (1) highly rated by PsyberGuide and (2) met the criteria for intervention features these participants indicated were desirable in a previous survey. Participants used the apps for 4 weeks and completed baseline and 4-week self-assessments of depression, anxiety emotional regulation, and suicide risk. Results: We found no differences between the apps in any outcome but did find significant changes in depression and anxiety over time (Patient Health Questionnaire [PHQ]-9: estimate=–1.5, SE 0.2, 95% CI –1.1 to –1.8, P<.001; Generalized Anxiety Disorder Scale [GAD]-7: estimate=–1.3, SE 0.2, 95% CI –1.0 to –1.6, P<.001). We found no significant changes in suicidal behavior (Suicide Behaviors Questionnaire-Revised [SBQ-R]) or emotional regulation (Difficulties in Emotion Regulation Scale – Short Form [DERS-SF]) for the 4 weeks. We did find a significant dose-response pattern for changes in depression and anxiety. Using the app at least once a week resulted in greater improvements in treatment conditions over time on depression (estimate=–0.6, SE 0.2, 95% CI 1.0-0.2, P=.003) and anxiety (estimate=0.1, SE 0.2, 95% CI 0.4-0.6, P=.78). There was no association between app frequency and changes in suicidal behavior (SBQ-R) or emotional regulation (DERS-SF). We further found a significant difference between the conditions with regard to app usability, with the control app being the most usable (meanBeautiful Mood 72.9, SD 16.7; meanCOVID Coach 71.2, SD 15.4; meanCalm 66.8, SD 17.3; mean7 Cups 65.2, SD 17.7). We found no significant differences for app acceptability or appropriateness. Conclusions: Few studies have evaluated prospectively the utility and usability of commercial apps for mood. This study found that free, self-guided commercial mobile mental health apps are seen as usable, but no one app is superior to the other. Although we found that regular use is indicated for effects on depression and anxiety to occur in those who are more symptomatic, regression to the mean cannot be ruled out. Trial Registration: ClinicalTrials.gov NCT04536935; https://tinyurl.com/mr36zx3s

Background: Smartphones are increasingly used in health research. They provide a continuous connection between participants and researchers to monitor long-term health trajectories of large populations at a fraction of the cost of traditional research studies. However, despite the potential of using smartphones in remote research, there is an urgent need to develop effective strategies to reach, recruit, and retain the target populations in a representative and equitable manner. Objective: We aimed to investigate the impact of combining different recruitment and incentive distribution approaches used in remote research on cohort characteristics and long-term retention. The real-world factors significantly impacting active and passive data collection were also evaluated. Methods: We conducted a secondary data analysis of participant recruitment and retention using data from a large remote observation study aimed at understanding real-world factors linked to cold, influenza, and the impact of traumatic brain injury on daily functioning. We conducted recruitment in 2 phases between March 15, 2020, and January 4, 2022. Over 10,000 smartphone owners in the United States were recruited to provide 12 weeks of daily surveys and smartphone-based passive-sensing data. Using multivariate statistics, we investigated the potential impact of different recruitment and incentive distribution approaches on cohort characteristics. Survival analysis was used to assess the effects of sociodemographic characteristics on participant retention across the 2 recruitment phases. Associations between passive data-sharing patterns and demographic characteristics of the cohort were evaluated using logistic regression. Results: We analyzed over 330,000 days of engagement data collected from 10,000 participants. Our key findings are as follows: first, the overall characteristics of participants recruited using digital advertisements on social media and news media differed significantly from those of participants recruited using crowdsourcing platforms (Prolific and Amazon Mechanical Turk; P<.001). Second, participant retention in the study varied significantly across study phases, recruitment sources, and socioeconomic and demographic factors (P<.001). Third, notable differences in passive data collection were associated with device type (Android vs iOS) and participants’ sociodemographic characteristics. Black or African American participants were significantly less likely to share passive sensor data streams than non-Hispanic White participants (odds ratio 0.44-0.49, 95% CI 0.35-0.61; P<.001). Fourth, participants were more likely to adhere to baseline surveys if the surveys were administered immediately after enrollment. Fifth, technical glitches could significantly impact real-world data collection in remote settings, which can severely impact generation of reliable evidence. Conclusions: Our findings highlight several factors, such as recruitment platforms, incentive distribution frequency, the timing of baseline surveys, device heterogeneity, and technical glitches in data collection infrastructure, that could impact remote long-term data collection. Combined together, these empirical findings could help inform best practices for monitoring anomalies during real-world data collection and for recruiting and retaining target populations in a representative and equitable manner.

Background: Although web-based mental health resources have the potential to assist millions, particularly those who face barriers to treatment, most mental health website visitors disengage before accessing resources that can help improve their mental health. Objective: We used a sequential multiple assignment randomized trial to test whether personalized tailoring improved engagement on a self-guided mental health website. Methods: Data were collected via voluntary response sampling on the Mental Health America website. Inclusion criteria included residing in the United States and viewing a postscreening survey after completing the Patient Health Questionnaire-9 (PHQ-9). Participants were randomized to 1 of 2 postscreening survey conditions: the demographics survey or the Next Steps survey, which included additional tailoring questions assessing perceived need and participants’ intended next steps. Participants who viewed the following screening results page were subsequently randomized to 1 of 5 conditions that displayed nontailored or tailored messages and featured resources, as well as persistent general resources that did not vary by condition. Data were analyzed using logistic regressions predicting disengagement and clicks on featured resources (versus persistent general resources) by condition. Results: Adding questions to inform tailoring significantly increased the odds of disengaging by 14% (demographics survey: 25%; Next Steps survey: 27.5%; odds ratio [OR] 1.14, 95% CI 1.11‐1.16; P <.001). Among participants who viewed a postscreening survey (n=169,647), 87,712 participants were randomized to the demographics survey condition, and 81,935 participants were randomized to the Next Steps survey condition. Among participants who submitted the demographics survey (n=38,490), tailoring resources to demographics reduced the odds of disengaging by 10% (OR 0.90, 95% CI 0.87‐0.94; P <.001) and, among those who engaged, increased the odds of clicking a featured resource versus a persistent general resource by 90% (OR 1.90, 95% CI 1.79‐2.01; P <.001). Among participants who submitted the Next Steps survey (n=34,204), tailoring messages to perceived need ( P =.33), tailoring resources to intended next steps ( P =.51), and a combination of both ( P =.52) did not significantly reduce the odds of disengaging compared with the nontailored condition. However, tailoring resources to intended next steps and combining a tailored message to perceived need with tailored resources to intended next steps increased the odds of clicking a featured resource by 25% (OR 1.25, 95% CI 1.14‐1.37; P <.001) and 34% (OR 1.34, 95% CI 1.23‐1.47; P <.001), respectively. Tailoring resources to demographics was significantly more effective in improving engagement than tailoring to perceived need or intended next steps ( P ≤.004). Conclusions: There was a small but statistically significant cost to engagement from adding tailoring questions assessing perceived need and intended next steps. Among the strategies tested in this study, tailoring resources to demographics was the most effective strategy for increasing engagement among visitors who viewed their screening results. This study demonstrates how personalization may increase engagement with mental health websites and provides design implications for future research.

Background: Although substantial progress has been made in establishing evidence-based psychosocial clinical interventions and implementation strategies for mental health, translating research into practice—particularly in more accessible, community settings—has been slow. Objective: This protocol outlines the renewal of the National Institute of Mental Health–funded University of Washington Advanced Laboratories for Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center, which draws from human-centered design (HCD) and implementation science to improve clinical interventions and implementation strategies. The Center’s second round of funding (2023-2028) focuses on using the Discover, Design and Build, and Test (DDBT) framework to address 3 priority clinical intervention and implementation strategy mechanisms (ie, usability, engagement, and appropriateness), which we identified as challenges to implementation and scalability during the first iteration of the center. Local redesign teams work collaboratively and share decision-making to carry out DDBT. Methods: All 4 core studies received institutional review board approval by June 2024, and each pilot project will pursue institutional review board approval when awarded. We will provide research infrastructure to 1 large effectiveness study and 3 exploratory pilot studies as part of the center grant. At least 4 additional small pilot studies will be solicited and funded by the center. All studies will explore the use of DDBT for clinical interventions and implementation strategies to identify modification targets to improve usability, engagement, and appropriateness in accessible nonspecialty settings (Discover phase); develop redesign solutions with local teams to address modification targets (Design and Build phase); and determine if redesign improves usability, engagement, and appropriateness (Test phase), as well as implementation outcomes. Center staff will collaborate with local redesign teams to develop and test clinical interventions and implementation strategies for community settings. We will collaborate with teams to use methods and centerwide measures that facilitate cross-project analysis of the effects of DDBT-driven redesign on outcomes of interest. Results: As of January 2025, three of the 4 core studies are underway. We will generate additional evidence on the robustness of DDBT and whether combining HCD and implementation science is an asset for improving clinical interventions and implementation strategies. Conclusions: During the first round of the center, we established that DDBT is a useful approach to systematically identify and address chronic challenges of implementing clinical interventions and implementation strategies. In this subsequent grant, we expect to increase evidence of DDBT’s impact on clinical interventions and implementation strategies by expanding a list of common challenges that could benefit from modification, a list of exemplary solutions to address these challenges, and guidance on using the DDBT framework. These resources will contribute to broader discourse on how to enhance implementation of clinical interventions and implementation strategies that integrate HCD and implementation science. International Registered Report Identifier (IRRID): PRR1-10.2196/65446

Background: Digital Mental Health (DMH) tools are an effective, readily accessible, and affordable form of mental health support. However, sustained engagement with DMH is suboptimal, with limited research on DMH engagement. The Health Action Process Approach (HAPA) is an empirically supported theory of health behavior adoption and maintenance. Whether this model also explains DMH tool engagement remains unknown. Objective: This study examined whether an adapted HAPA model predicted engagement with DMH via a self-guided website. Methods: Visitors to the Mental Health America (MHA) website were invited to complete a brief survey measuring HAPA constructs. This cross-sectional study tested the adapted HAPA model with data collected using voluntary response sampling from 16,078 sessions (15,619 unique IP addresses from United States residents) on the MHA website from October 2021 through February 2022. Model fit was examined via structural equation modeling in predicting two engagement outcomes: (1) choice to engage with DMH (ie, spending 3 or more seconds on an MHA page, excluding screening pages) and (2) level of engagement (ie, time spent on MHA pages and number of pages visited, both excluding screening pages). Results: Participants chose to engage with the MHA website in 94.3% (15,161/16,078) of the sessions. Perceived need (β=.66; P<.001), outcome expectancies (β=.49; P<.001), self-efficacy (β=.44; P<.001), and perceived risk (β=.17-.18; P<.001) significantly predicted intention, and intention (β=.77; P<.001) significantly predicted planning. Planning was not significantly associated with choice to engage (β=.03; P=.18). Within participants who chose to engage, the association between planning with level of engagement was statistically significant (β=.12; P<.001). Model fit indices for both engagement outcomes were poor, with the adapted HAPA model accounting for only 0.1% and 1.4% of the variance in choice to engage and level of engagement, respectively. Conclusions: Our data suggest that the HAPA model did not predict engagement with DMH via a self-guided website. More research is needed to identify appropriate theoretical frameworks and practical strategies (eg, digital design) to optimize DMH tool engagement.

Background: Social media and search engine tech companies, such as Google, use the wealth of data at their disposal to create algorithms to predict suicide risk and provide interventions based on company policies and outputs from these algorithms. Neither the details of these algorithms nor their accuracy in risk prediction are shared with the public; thus, research is needed to verify whether algorithms applied to search engine data can accurately predict suicide risk in a platform user. This paper provides new information about the methodological challenges in studying the value of search engine data for suicide prediction. Objective: This paper describes the study methods and experiences regarding recruiting and retaining participants in a year-long, fully remote prospective cohort study of suicide prediction in people with variable risk of suicide, using Google Search data. Methods: We aimed to recruit 1000 United States–dwelling and English-speaking adults with current suicidal ideation who fell into 1 of 3 categories: participants with a suicide attempt in the past year, participants with a lifetime suicide attempt but not one in the past year, and participants with no suicide attempt in their lifetime. Recruitment sources included online advertisements hosted on various websites, to differing levels of success. Participants completed online surveys and an interview via Zoom to enroll in the study and then were sent follow-up surveys every 2 weeks for the year following initial enrollment. Recruitment was conducted from March 2022 to January 2024, with follow-up data being collected through February 2025. Results: Of 15,120 records created in our Research Electronic Data Capture (REDCap) survey system, 485 participants were eligible to participate in the prospective year of the study. Recruitment challenges were mainly due to participant dropout across all enrollment time points, with 63.7% (9636/15,120) of the prescreening sample dropping out or refusing. Other issues included the difficulty of downloading Google Search data (446/3277, 13.6%) and not having enough data to meet the inclusion criteria among those who were able to successfully donate (252/3277, 7.7%). Lastly, the study encountered fraudulent behavior, with 489 of 15,120 records (3.2%) being identified as fraudulent over the course of the study. Conclusions: Despite past success with remote clinical research methods, we found that recruitment can be significantly hampered by fraudulent participation, despite the use of state-of-the-art methods to mitigate this problem. Further, the ability to acquire useful Google Search data is extremely challenging and necessitates the use of technical assistance and continuous monitoring of Google Takeout data structure to ensure accurate and complete data donations. The solutions we provide for this type of research should be helpful for future endeavors. However, research is still limited by the barriers to obtaining data from technology companies. These limitations have significant implications in detecting the accuracy and utility of suicide prediction from search engine data.