Scientists and practitioners alike need reliable, valid measures of contextual factors that influence implementation. Yet, few existing measures demonstrate reliability or validity. To meet this need, we developed and assessed the psychometric properties of measures of several constructs within the Inner Setting domain of the Consolidated Framework for Implementation Research (CFIR). We searched the literature for existing measures for the 7 Inner Setting domain constructs (Culture Overall, Culture Stress, Culture Effort, Implementation Climate, Learning Climate, Leadership Engagement, and Available Resources). We adapted items for the healthcare context, pilot-tested the adapted measures in 4 Federally Qualified Health Centers (FQHCs), and implemented the revised measures in 78 FQHCs in the 7 states (N = 327 respondents) with a focus on colorectal cancer (CRC) screening practices. To psychometrically assess our measures, we conducted confirmatory factor analysis models (CFA; structural validity), assessed inter-item consistency (reliability), computed scale correlations (discriminant validity), and calculated inter-rater reliability and agreement (organization-level construct reliability and validity). CFAs for most constructs exhibited good model fit (CFI > 0.90, TLI > 0.90, SRMR < 0.08, RMSEA < 0.08), with almost all factor loadings exceeding 0.40. Scale reliabilities ranged from good (0.7 ≤ α < 0.9) to excellent (α ≥ 0.9). Scale correlations fell below 0.90, indicating discriminant validity. Inter-rater reliability and agreement were sufficiently high to justify measuring constructs at the clinic-level. Our findings provide psychometric evidence in support of the CFIR Inner Setting measures. Our findings also suggest the Inner Setting measures from individuals can be aggregated to represent the clinic-level. Measurement of the Inner Setting constructs can be useful in better understanding and predicting implementation in FQHCs and can be used to identify targets of strategies to accelerate and enhance implementation efforts in FQHCs.

The failure rates for implementing complex innovations in healthcare organizations are high. Estimates range from 30% to 90% depending on the scope of the organizational change involved, the definition of failure, and the criteria to judge it. The innovation implementation framework offers a promising approach to examine the organizational factors that determine effective implementation. To date, the utility of this framework in a healthcare setting has been limited to qualitative studies and/or group level analyses. Therefore, the goal of this study was to quantitatively examine this framework among individual participants in the National Cancer Institute’s Community Clinical Oncology Program using structural equation modeling. We examined the innovation implementation framework using structural equation modeling (SEM) among 481 physician participants in the National Cancer Institute’s Community Clinical Oncology Program (CCOP). The data sources included the CCOP Annual Progress Reports, surveys of CCOP physician participants and administrators, and the American Medical Association Physician Masterfile. Overall the final model fit well. Our results demonstrated that not only did perceptions of implementation climate have a statistically significant direct effect on implementation effectiveness, but physicians’ perceptions of implementation climate also mediated the relationship between organizational implementation policies and practices (IPP) and enrollment (p <0.05). In addition, physician factors such as CCOP PI status, age, radiological oncologists, and non-oncologist specialists significantly influenced enrollment as well as CCOP organizational size and structure, which had indirect effects on implementation effectiveness through IPP and implementation climate. Overall, our results quantitatively confirmed the main relationship postulated in the innovation implementation framework between IPP, implementation climate, and implementation effectiveness among individual physicians. This finding is important, as although the model has been discussed within healthcare organizations before, the studies have been predominately qualitative in nature and/or at the organizational level. In addition, our findings have practical applications. Managers looking to increase implementation effectiveness of an innovation should focus on creating an environment that physicians perceive as encouraging implementation. In addition, managers should consider instituting specific organizational IPP aimed at increasing positive perceptions of implementation climate. For example, IPP should include specific expectations, support, and rewards for innovation use.

The National Diabetes Prevention Program (National DPP) is rapidly expanding in an effort to help those at high risk of type 2 diabetes prevent or delay the disease. In 2012, the Centers for Disease Control and Prevention funded six national organizations to scale and sustain multistate delivery of the National DPP lifestyle change intervention (LCI). This study aims to describe reach, adoption, and maintenance during the 4-year funding period and to assess associations between site-level factors and program effectiveness regarding participant attendance and participation duration. The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework was used to guide the evaluation from October 2012 to September 2016. Multilevel linear regressions were used to examine associations between participant-level demographics and site-level strategies and number of sessions attended, attendance in months 7–12, and duration of participation. The six funded national organizations increased the number of participating sites from 68 in 2012 to 164 by 2016 across 38 states and enrolled 14,876 eligible participants. By September 2016, coverage for the National DPP LCI was secured for 42 private insurers and 7 public payers. Nearly 200 employers were recruited to offer the LCI on site to their employees. Site-level strategies significantly associated with higher overall attendance, attendance in months 7–12, and longer participation duration included using self-referral or word of mouth as a recruitment strategy, providing non-monetary incentives for participation, and using cultural adaptations to address participants’ needs. Sites receiving referrals from healthcare providers or health systems also had higher attendance in months 7–12 and longer participation duration. At the participant level, better outcomes were achieved among those aged 65+ (vs. 18–44 or 45–64), those who were overweight (vs. obesity), those who were non-Hispanic white (vs. non-Hispanic black or multiracial/other races), and those eligible based on a blood test or history of gestational diabetes mellitus (vs. screening positive on a risk test). In a time of rapid dissemination of the National DPP LCI the findings of this evaluation can be used to enhance program implementation and translate lessons learned to similar organizations and settings.

Background: Our purpose was to identify physicians’ individual characteristics, attitudes, and organizational contextual factors associated with higher enrollment of patients in cancer clinical trials among physician participants in the National Cancer Institute’s Community Clinical Oncology Program (CCOP). We hypothesized that physicians’ individual characteristics, such as age, medical specialty, tenure, CCOP organizational factors (i.e. policies and procedures to encourage enrollment), and attitudes toward participating in CCOP would directly determine enrollment. We also hypothesized that physicians’ characteristics and CCOP organizational factors would influence physicians’ attitudes toward participating in CCOP, which in turn would predict enrollment. Methods: We evaluated enrollment in National Cancer Institute–sponsored cancer clinical trials in 2011 among 481 physician participants using Structural Equation Modeling. The data sources include CCOP Annual Progress Reports, two surveys of CCOP administrators and physician participants, and the American Medical Association Masterfile. Results: Physicians with more positive attitudes toward participating in CCOP enrolled more patients than physicians with less positive attitudes. In addition, physicians who practiced in CCOPs that had more supportive policies and practices in place to encourage enrollment (i.e. offered trainings, provided support to screen and enroll patients, gave incentives to enroll patients, instituted minimum accrual expectations) also significantly enrolled more patients. Physician status as CCOP Principal Investigator had a positive direct effect on enrollment, while physician age and non-oncology medical specialty had negative direct effects on enrollment. Neither physicians’ characteristics nor CCOP organizational factors indirectly influenced enrollment through an effect on physician attitudes. Conclusion: We examined whether individual physicians’ characteristics and attitudes, as well as CCOP organizational factors, influenced patient enrollment in cancer clinical trials among CCOP physicians. Physician attitudes and CCOP organizational factors had positive direct effects, but not indirect effects, on physician enrollment of patients. Our results could be used to develop physician-directed strategies aimed at increasing involvement in clinical research. For example, administrators may want to ensure physicians have access to support staff to help screen and enroll patients or institute minimum accrual expectations. Our results also highlight the importance of recruiting physicians for volunteer clinical research programs whose attitudes and values align with programmatic goals. Given that physician involvement is a key determinant of patient enrollment in clinical trials, these interventions could expand the overall number of patients involved in cancer research. These strategies will be increasingly important as the CCOP network continues to evolve.

Hydroxyurea reduces sickle cell disease (SCD) complications, but medication adherence is low. We tested 2 mobile health (mHealth) interventions targeting determinants of low adherence among patients (InCharge Health) and low prescribing among providers (HU Toolbox) in a multi-center, non-randomized trial of individuals with SCD ages 15-45. We compared the percentage of days covered (PDC), labs, healthcare utilization, and self-reported pain over 24 weeks of intervention and 12 weeks post-study with a 24-week preintervention interval. We enrolled 293 patients (51% male; median age 27.5 years, 86.8% HbSS/HbSβ0-thalassemia). The mean change in PDC among 235 evaluable subjects increased (39.7% to 56.0%; P < 0.001) and sustained (39.7% to 51.4%, P < 0.001). Mean HbF increased (10.95% to 12.78%; P = 0.03). Self-reported pain frequency reduced (3.54 to 3.35 events/year; P = 0.041). InCharge Health was used ≥1 day by 199 of 235 participants (84.7% implementation; median usage: 17% study days; IQR: 4.8-45.8%). For individuals with ≥1 baseline admission for pain, admissions per 24 weeks declined from baseline through 24 weeks (1.97 to 1.48 events/patient, P = 0.0045) and weeks 25-36 (1.25 events/patient, P = 0.0015). PDC increased with app use (P < 0.001), with the greatest effect in those with private insurance (P = 0.0078), older subjects (P = 0.033), and those with lower pain interference (P = 0.0012). Of the 89 providers (49 hematologists, 36 advanced care providers, 4 unreported), only 11.2% used HU Toolbox ≥1/month on average. This use did not affect change in PDC. Tailoring mHealth solutions to address barriers to hydroxyurea adherence can potentially improve adherence and provide clinical benefits. A definitive randomized study is warranted. This trial was registered at www.clinicaltrials.gov as #NCT04080167.

Background: Hydroxyurea therapy is effective for reducing complications related to sickle cell disease (SCD) and is recommended by National Health Lung and Blood Institute care guidelines. However, hydroxyurea is underutilized, and adherence is suboptimal. We wanted to test a multilevel mobile health (mHealth) intervention to increase hydroxyurea adherence among patients and improve prescribing among providers in a multicenter clinical trial. In the first 2 study sites, participants were exposed to the early phases of the COVID-19 pandemic, which included disruption to their regular SCD care. Objective: We aimed to describe the impact of the COVID-19 pandemic on the implementation of an mHealth behavioral intervention for improving hydroxyurea adherence among patients with SCD. Methods: The first 2 sites initiated enrollment 3 months prior to the start of the pandemic (November 2019 to March 2020). During implementation, site A clinics shut down for 2 months and site B clinics shut down for 9 months. We used the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework to evaluate the implementation and effectiveness of the intervention. mHealth implementation was assessed based on patients’ daily app use. Adherence to hydroxyurea was calculated as the proportion of days covered (PDC) from prescription records over the first 12 and 24 weeks after implementation. A linear model examined the relationship between app usage and PDC change, adjusting for baseline PDC, lockdown duration, and site. We conducted semistructured interviews with patients, health care providers, administrators, and research staff to identify factors associated with mHealth implementation and effectiveness. We used a mixed methods approach to investigate the convergence of qualitative and quantitative findings. Results: The percentage of patients accessing the app decreased after March 15, 2020 from 86% (n=55) to 70% (n=45). The overall mean PDC increase from baseline to week 12 was 4.5% (P=.32) and to week 24 was 1.5% (P=.70). The mean PDC change was greater at site A (12 weeks: 20.9%; P=.003; 24 weeks: 16.7%; P=.01) than site B (12 weeks: −8.2%; P=.14; 24 weeks: −10.3%; P=.02). After adjustment, PDC change was 13.8% greater in those with increased app use after March 15, 2020. Interview findings indicated that site B’s closure during COVID-19 had a greater impact, but almost all patients reported that the InCharge Health app helped support more consistent medication use. Conclusions: We found significant impacts of the early clinic lockdowns, which reduced implementation of the mHealth intervention and led to reduced patient adherence to hydroxyurea. However, disruptions were lower among participants who experienced shorter clinic lockdowns and were associated with higher hydroxyurea adherence. Investigation of added strategies to mitigate the effects of care interruptions during major emergencies (eg, patient coaching and health navigation) may “insulate” the implementation of interventions to increase medication adherence. Trial Registration: ClinicalTrials.gov NCT04080167; https://clinicaltrials.gov/ct2/show/NCT04080167 International Registered Report Identifier (IRRID): RR2-10.2196/16319