Scientists and practitioners alike need reliable, valid measures of contextual factors that influence implementation. Yet, few existing measures demonstrate reliability or validity. To meet this need, we developed and assessed the psychometric properties of measures of several constructs within the Inner Setting domain of the Consolidated Framework for Implementation Research (CFIR). We searched the literature for existing measures for the 7 Inner Setting domain constructs (Culture Overall, Culture Stress, Culture Effort, Implementation Climate, Learning Climate, Leadership Engagement, and Available Resources). We adapted items for the healthcare context, pilot-tested the adapted measures in 4 Federally Qualified Health Centers (FQHCs), and implemented the revised measures in 78 FQHCs in the 7 states (N = 327 respondents) with a focus on colorectal cancer (CRC) screening practices. To psychometrically assess our measures, we conducted confirmatory factor analysis models (CFA; structural validity), assessed inter-item consistency (reliability), computed scale correlations (discriminant validity), and calculated inter-rater reliability and agreement (organization-level construct reliability and validity). CFAs for most constructs exhibited good model fit (CFI > 0.90, TLI > 0.90, SRMR < 0.08, RMSEA < 0.08), with almost all factor loadings exceeding 0.40. Scale reliabilities ranged from good (0.7 ≤ α < 0.9) to excellent (α ≥ 0.9). Scale correlations fell below 0.90, indicating discriminant validity. Inter-rater reliability and agreement were sufficiently high to justify measuring constructs at the clinic-level. Our findings provide psychometric evidence in support of the CFIR Inner Setting measures. Our findings also suggest the Inner Setting measures from individuals can be aggregated to represent the clinic-level. Measurement of the Inner Setting constructs can be useful in better understanding and predicting implementation in FQHCs and can be used to identify targets of strategies to accelerate and enhance implementation efforts in FQHCs.

Pharmacies are promising alternative settings for human papillomavirus (HPV) vaccination because of their population reach, convenience, and existing infrastructure for vaccine delivery. However, pharmacies in the US are rarely used for adolescent HPV vaccination. We sought to document challenges and opportunities of implementing pharmacy-located HPV vaccination services in five US states by mapping process evaluation results onto key implementation science constructs: service penetration, acceptability, appropriateness, feasibility, fidelity, adoption, and sustainability. Pilot projects were planned in North Carolina (k=2 pharmacies), Michigan (k=10), Iowa (k=2), Kentucky (k=1), and Oregon (no pharmacy recruited) with varying procedures and recruitment strategies. Sites had open enrollment for a combined 12 months. Despite substantial efforts in these states, only 13 HPV vaccine doses were administered to adolescents and three doses to age-eligible young adults. We identified two major reasons for these underperforming results. First, poor outcomes on service penetration and appropriateness pointed to engagement barriers: low parent demand and engagement among pharmacy staff. Second, poor outcomes on feasibility, adoption, and sustainability appeared to result from administrative hurdles: lacking third party reimbursement (i.e., billing commercial payers, participation in Vaccines for Children program) and limited integration into primary care systems. In summary, pilot projects in five states all struggled to administer HPV vaccines. Opportunities for making pharmacies a successful setting for adolescent HPV vaccination include expanding third party reimbursement to cover all vaccines administered by pharmacists, increasing public awareness of pharmacists’ immunization training, and improving care coordination with primary care providers.