Given the importance of emotions in psychotherapy, valid measures are essential for research and practice. As emotions are expressed at different levels, multimodal measurements are needed for a nuanced assessment. Natural Language Processing (NLP) could augment the measurement of emotions. The study explores the validity of sentiment analysis in psychotherapy transcripts. We used a transformer-based NLP algorithm to analyze sentiments in 85 transcripts from 35 patients. Construct and criterion validity were evaluated using self- and therapist reports and process and outcome measures via correlational, multitrait-multimethod, and multilevel analyses. The results provide indications in support of the sentiments' validity. For example, sentiments were significantly related to self- and therapist reports of emotions in the same session. Sentiments correlated significantly with in-session processes (e.g., coping experiences), and an increase in positive sentiments throughout therapy predicted better outcomes after treatment termination. Sentiment analysis could serve as a valid approach to assessing the emotional tone of psychotherapy sessions and may contribute to the multimodal measurement of emotions. Future research could combine sentiment analysis with automatic emotion recognition in facial expressions and vocal cues via the Nonverbal Behavior Analyzer (NOVA). Limitations (e.g., exploratory study with numerous tests) and opportunities are discussed.
29 March 2024 in Administration and Policy in Mental Health and Mental Health Services Research
We aim to use topic modeling, an approach for discovering clusters of related words (“topics”), to predict symptom severity and therapeutic alliance in psychotherapy transcripts, while also identifying the most important topics and overarching themes for prediction. We analyzed 552 psychotherapy transcripts from 124 patients. Using BERTopic (Grootendorst, 2022), we extracted 250 topics each for patient and therapist speech. These topics were used to predict symptom severity and alliance with various competing machine-learning methods. Sensitivity analyses were calculated for a model based on 50 topics, LDA-based topic modeling, and a bigram model. Additionally, we grouped topics into themes using qualitative analysis and identified key topics and themes with eXplainable Artificial Intelligence (XAI). Symptom severity could be predicted with highest accuracy by patient topics ( $$r$$ =0.45, 95%-CI 0.40, 0.51), whereas alliance was better predicted by therapist topics ( $$r$$ =0.20, 95%-CI 0.16, 0.24). Drivers for symptom severity were themes related to health and negative experiences. Lower alliance was correlated with various themes, especially psychotherapy framework, income, and everyday life. This analysis shows the potential of using topic modeling in psychotherapy research allowing to predict several treatment-relevant metrics with reasonable accuracy. Further, the use of XAI allows for an analysis of the individual predictive value of topics and themes. Limitations entail heterogeneity across different topic modeling hyperparameters and a relatively small sample size.
Rating scales have shaped psychological research, but are resource-intensive and can burden participants. Large Language Models (LLMs) offer a tool to assess latent constructs in text. This study introduces LLM rating scales, which use LLM responses instead of human ratings. We demonstrate this approach with an LLM rating scale measuring patient engagement in therapy transcripts. Automatically transcribed videos of 1,131 sessions from 155 patients were analyzed using DISCOVER, a software framework for local multimodal human behavior analysis. Llama 3.1 8B LLM rated 120 engagement items, averaging the top eight into a total score. Psychometric evaluation showed a normal distribution, strong reliability (ω = 0.953), and acceptable fit (CFI = 0.968, SRMR = 0.022), except RMSEA = 0.108. Validity was supported by significant correlations with engagement determinants (e.g., motivation, r = .413), processes (e.g., between-session efforts, r = .390), and outcomes (e.g., symptoms, r = − .304). Results remained robust across bootstrap resampling and cross-validation, accounting for nested data. The LLM rating scale exhibited strong psychometric properties, demonstrating the potential of the approach as an assessment tool. Importantly, this automated approach uses interpretable items, ensuring clear understanding of measured constructs, while supporting local implementation and protecting confidential data.
Background and aims: For the first time, the ICD-11 provides the diagnosis compulsive sexual behavior disorder (CSBD) that can be assigned for pornography use disorder (PUD). This study aimed to estimate the prevalence of PUD and associated consequences in Germany, to identify the psychotherapy demand among likely PUD (lPUD) cases and the treatment supply in different psychotherapeutic settings, to survey psychotherapists' level of expertise regarding PUD, and to identify predictors for psychotherapy demand.Methods: Four studies were conducted: 1. Online study in the general population (n = 2070; m = 48.9%, f = 50.8%, d = 0.2%), 2. Survey among practicing psychotherapists (n = 983), 3. Survey of psychotherapists in psychotherapeutic outpatient clinics (n = 185), 4. Interviews with psychotherapeutic inpatient clinics (n = 28).Results: The estimated prevalence of lPUD in the online study was 4.7% and men were 6.3 times more often affected than women. Compared to individuals without PUD, individuals with lPUD more often indicated negative consequences in performance-related areas. Among lPUD cases, 51.2% of men and 64.3% of women were interested in a specialized PUD treatment. Psychotherapists reported 1.2%–2.9% of lPUD cases among their patients. 43.2%–61.5% of psychotherapists stated to be poorly informed about PUD. Only 7% of psychotherapeutic inpatient clinics provided specific treatments to patients with PUD. While, among other factors, negative consequences attributed to lPUD were predictive for psychotherapy demand, weekly pornography consumption, subjective well-being, and religious attachment were not.Discussion and conclusions: Although PUD occurs quite often in Germany, availability of mental health care services for PUD is poor. Specific PUD treatments are urgently needed.
15 December 2023 in Administration and Policy in Mental Health and Mental Health Services Research
Outcome measurement including data-informed decision support for therapists in psychological therapy has developed impressively over the past two decades. New technological developments such as computerized data assessment, and feedback tools have facilitated advanced implementation in several seetings. Recent developments try to improve the clinical decision-making process by connecting clinical practice better with empirical data. For example, psychometric data can be used by clinicians to personalize the selection of therapeutic programs, strategies or modules and to monitor a patient’s response to therapy in real time. Furthermore, clinical support tools can be used to improve the treatment for patients at risk for a negative outcome. Therefore, measurement-based care can be seen as an important and integral part of clinical competence, practice, and training. This is comparable to many other areas in the healthcare system, where continuous monitoring of health indicators is common in day-to-day clinical practice (e.g., fever, blood pressure). In this paper, we present the basic concepts of a data-informed decision support system for tailoring individual psychological interventions to specific patient needs, and discuss the implications for implementing this form of precision mental health in clinical practice.
Although psychotherapy has shown to be effective for most patients, about one-third of patients do not benefit or deteriorate during treatment. Technical progress has allowed the integration of routine outcome monitoring (ROM) into treatment, helping therapists detect patients at risk for a nonresponse or poor outcome early on. Psychological therapy can be enhanced by providing therapists with individual treatment recommendations for these at-risk patients. One example of such a comprehensive feedback system is the Trier Treatment Navigator (TTN). This clinical case study aims to illustrate the implementation of the TTN in a cognitive behavioral therapy (CBT) outpatient clinic in the treatment of a 30-year-old patient called Ms. Daun, who has a recurrent depressive disorder. Based on this case, the benefits of applying information from ROM in psychotherapy, important context factors, and possible implementation issues are discussed. We conclude by encouraging practitioners to integrate ROM into their clinical thinking and daily practice.