The relationship between the higher-order factor model and the hierarchical factor model is explored formally. We show that the Schmid-Leiman transformation produces constrained hierarchical factor solutions. Using a generalized Schmid-Leiman transformation and its inverse, we show that for any unconstrained hierarchical factor model there is an equivalent higher-order factor model with direct effects (loadings) on the manifest variables from the higher-order factors. Therefore, the class of higher-order factor models (without direct effects of higher-order factors) is nested within the class of unconstrained hierarchical factor models. In light of these formal results, we discuss some implications for testing the higher-order factor model and the issue of general factor. An interesting aspect concerning the efficient fitting of the higher-order factor model with direct effects is noted.
1 April 2011 in Expert Review of Pharmacoeconomics & Outcomes Research
In recent years, the US FDA has become more critical of instruments used to measure patient-reported outcomes (PROs) in clinical trials. To facilitate decisions related to the approval of drugs, la...
30 July 2019 in Journal of Patient-Reported Outcomes
This paper is part of a series comparing different psychometric approaches to evaluate patient-reported outcome (PRO) measures using the same items and dataset. We provide an overview and example application to demonstrate 1) using item response theory (IRT) to identify poor and well performing items; 2) testing if items perform differently based on demographic characteristics (differential item functioning, DIF); and 3) balancing IRT and content validity considerations to select items for short forms. Model fit, local dependence, and DIF were examined for 51 items initially considered for the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Depression item bank. Samejima’s graded response model was used to examine how well each item measured severity levels of depression and how well it distinguished between individuals with high and low levels of depression. Two short forms were constructed based on psychometric properties and consensus discussions with instrument developers, including psychometricians and content experts. Calibrations presented here are for didactic purposes and are not intended to replace official PROMIS parameters or to be used for research. Of the 51 depression items, 14 exhibited local dependence, 3 exhibited DIF for gender, and 9 exhibited misfit, and these items were removed from consideration for short forms. Short form 1 prioritized content, and thus items were chosen to meet DSM-V criteria rather than being discarded for lower discrimination parameters. Short form 2 prioritized well performing items, and thus fewer DSM-V criteria were satisfied. Short forms 1–2 performed similarly for model fit statistics, but short form 2 provided greater item precision. IRT is a family of flexible models providing item- and scale-level information, making it a powerful tool for scale construction and refinement. Strengths of IRT models include placing respondents and items on the same metric, testing DIF across demographic or clinical subgroups, and facilitating creation of targeted short forms. Limitations include large sample sizes to obtain stable item parameters, and necessary familiarity with measurement methods to interpret results. Combining psychometric data with stakeholder input (including people with lived experiences of the health condition and clinicians) is highly recommended for scale development and evaluation.
7 September 2012 in The American Journal of Sports Medicine
Background: The Knee Injury and Osteoarthritis Outcome Score (KOOS) assesses acute and chronic knee injuries or early-onset osteoarthritis in young, active patients. The United States Food and Drug Administration guidelines recommend that patient-reported outcome instruments used to support clinical trial label claims should demonstrate content validity using patient input and have acceptable psychometric properties in the target population. To use the KOOS subscales in safety and efficacy trials assessing new treatments for patients with articular cartilage lesions, additional validation work, using input from patients with articular cartilage lesions, was necessary. Purpose: Qualitative and quantitative evaluations of the KOOS subscales’ validity among patients with articular cartilage lesions were conducted to support their use as clinically meaningful end points in clinical trials. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: For qualitative analysis, cognitive interviews involving concept elicitation and cognitive debriefing with the KOOS items were conducted with 15 participants aged 25 to 52 years. Participants either were candidates for cartilage repair or had undergone cartilage repair 6 months or more before the study. For the quantitative analysis, a psychometric evaluation of the KOOS was conducted with clinical trial data from 54 patients, aged 18 to 55 years, evaluating the Cartilage Autograft Implantation System in the United States (n = 29) and the European Union (n = 25). Data were collected before surgery and at 7 postsurgical visits up to 12 months. Internal consistency and test-retest reliability, construct validity, responsiveness, and estimates of the minimal detectable change (MDC) were assessed. Test-retest reliability was assessed using data from months 2 and 3 on a subset of stable patients. Results: Qualitative research confirmed that concepts measured on the KOOS are important to patients with articular cartilage lesions. Most participants reported the KOOS was comprehensive and appropriate. In the quantitative research, KOOS subscales showed excellent internal consistency reliability (range, .74-.97 at baseline) and test-retest reliability (range, .78-.82). Construct validity results supported hypothesized relationships, with significant correlations (r ≥ .50) in the expected directions. Responsiveness analyses demonstrated excellent sensitivity to change; standardized response means ranged from 0.8 to 1.2, and MDC estimates ranged from 7.4 to 12.1. Conclusion: The study results support the use of the KOOS subscales among patients with articular cartilage lesions.