Despite decades of policy emphasizing the role of evidence in guiding services, few studies have sought to improve the degree to which evidence is used in supervision and treatment. This study reports supervisor and therapist outcomes from the Reaching Families multisite cluster-randomized controlled trial, which tested the effects of a coordinated knowledge system (CKS) against practice guidelines (PG) on the use of evidence in supervision and treatment targeting low treatment engagement in publicly funded youth community mental health organizations located in two geographically distinct, underresourced communities where service inequities are common. The sample included 121 mental health professionals (92.6% female; 81.0% Black, Indigenous, and people of color1) randomly assigned to a CKS or PG control condition. We recorded, transcribed, and coded 430 supervision and 208 treatment sessions involving 221 youth (Mage = 13.1 years, 46.2% female; 78.7% Black, Indigenous, and people of color) and/or their caregivers who reported engagement concerns during therapy. CKS dyads showed uniformly greater use of evidence focused on specific client needs relative to dyads in the PG condition, with large effect sizes and no differences in the effect of condition across the sites. Secondary analyses showed that tools in the CKS condition were perceived significantly more positively than those in the PG condition in terms of effort and effectiveness, and supervisory workload was the same across both conditions. In routine clinical care delivered within highly representative community settings, a strategically designed knowledge resource can improve evidence-based reasoning and action and be perceived as easy to use and useful without negatively impacting workload. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

The clinical decisions and actions of evidence-based practice in psychology (EBPP) are largely underspecified and poorly understood, in part due to the lack of measurement methods. We tested the reliability of a behavioral coding system that characterizes a flow of interrelated activities that includes problem detection and prioritization, intervention selection and implementation, and review of intervention integrity and impact. The context included two publicly funded youth mental health service organizations located in geographically distinct and underresourced communities in the U.S. where service inequities are common. We sampled 84 digitally recorded and transcribed supervision events that included a sample of professionals who were mostly women (93.02%) and BIPOC (86.04%) whose self-reported race/ethnicity matched the youth populations they served. We coded these events for activities (e.g., considering) and their predicate content (i.e., problems or practices) and examined reliability of these codes applied to excerpts (i.e., small contiguous units of dialogue) as well as to complete events. Interrater reliability estimates showed that, overall, coders reliably rated the occurrence and extensiveness of activities and content. Excerpt coding was generally more reliable than event coding. However, mathematical aggregation of excerpt coding offered a superior method for estimating event codes reliably, reducing individual subjectivity while providing event level synthesis of activities that are grounded in excerpt level details. The assessment of clinical decisions and actions has the potential to unpack the black box of EBPP, with different methods best suited to different research questions and resource considerations.

This qualitative study aimed to understand the intrapersonal, interpersonal, organizational, community, and public policy resources that peer support workers (PSWs) in the substance use disorder (SUD) recovery field perceive as helpful in supporting their efforts. Twenty-five PSWs participated in semistructured interviews designed to elicit their experiences and perspectives on resources that currently support or could better support their work. PSWs had a mean age of 49.3 years (SD = 12.9) and were primarily cisgender women (n = 16, 64%) and White (n = 20, 80%). Interviews were audio-recorded, transcribed, and analyzed using rapid qualitative analysis. To ensure validity, a PSW consultant reviewed the study materials and themes. At the intrapersonal level, PSWs employed a variety of self-care practices, such as mindfulness, faith-based activities, attendance at recovery meetings, and boundary setting at work. At the interpersonal level, PSWs reported a strong network of social capital, including friends, family, and supervisors. At the organizational level, PSWs identified supportive resources, including employment benefits, a positive work climate, and workplace autonomy. At the community level, PSWs leveraged networks with local organizations. At the public policy level, PSWs appreciated support from government officials. Desired resources included more time for self-care, empathy and understanding from family and non-PSW coworkers, reduced logistical burdens and more supervision, community resources to address social determinants of health, and reduced stigma toward people with SUDs and greater funding. By reinforcing current resources and introducing desired resources, organizations, community members, and policymakers can build the resilience of PSWs and promote high-quality services for the prevention and treatment of SUDs. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Service integrity, or the comparison of what has occurred with what was expected to occur, is essential to managing service quality. This paper tested the reliability of an expanded set of integrity metrics that included targets (i.e. problems targeted in intervention), practices (i.e. clinical procedures), and steps (i.e. sub-components of practices) measured at different levels of dialogue and across different types of service events. This study was conducted in two publicly funded youth mental health systems located in geographically distinct, under-resourced communities where service inequities are common. Targets, practices, and steps were coded in 78 treatment events and 84 clinical supervision events that were digitally recorded and transcribed. Fleiss' kappa (K) and intraclass correlation coefficients (ICCs) yielded estimates for the interrater reliability of codes in excerpts (i.e. small contiguous units of dialogue) as well as complete events (i.e. entire supervision or treatment sessions). Coders generally detected targets, practices, and steps in supervision and treatment with acceptable reliability. On average, coders detected steps more reliably than practices, targets more reliably in excerpts than in events, and supervision content more reliably than treatment content. These findings demonstrate the potential to reliably code a uniform set of metrics for expanding the analysis of service integrity in dynamic care settings. Assessing the extent to which delivered practices match targets, delivered treatment matches supervision discussions, and delivered steps reflect the practices intended to be delivered can yield crucial insights for improving service coordination, continuity, and overall quality of mental health care.