Background Socioeconomic status (SES) is an important determinant of health, but SES measures are frequently unavailable in commonly used datasets. Area-level SES measures are used as proxy measures of individual SES when the individual measures are lacking. Little is known about the agreement between individual-level versus area-level SES measures in mixed urban–rural settings. Methods We identified SES agreement by comparing information from telephone self-reported SES levels and SES calculated from area-level SES measures. We assessed the impact of this agreement on reported associations between SES and rates of childhood obesity, low birth weight <2500 g and smoking within the household in a mixed urban–rural setting. Results 750 households were surveyed with a response rate of 62%: 51% male, 89% Caucasian; mean child age 9.5 years. Individual-level self-reported income was more strongly associated with all three childhood health outcomes compared to area-level SES. We found significant disagreement rates of 22–31%. The weighted Cohen's κ indices ranged from 0.15 to 0.22, suggesting poor agreement between individual-level and area-level measures. Conclusion In a mixed urban–rural setting comprised of both rural and urbanised areas, area-level SES proxy measures significantly disagree with individual SES measures, and have different patterns of association with health outcomes from individual-level SES measures. Area-level SES may be an unsuitable proxy for SES when individual rather than community characteristics are of primary concern.
9 April 2022 in Journal of the American Medical Informatics Association
Objective: Artificial intelligence (AI) models may propagate harmful biases in performance and hence negatively affect the underserved. We aimed to assess the degree to which data quality of electronic health records (EHRs) affected by inequities related to low socioeconomic status (SES), results in differential performance of AI models across SES. Materials and Methods: This study utilized existing machine learning models for predicting asthma exacerbation in children with asthma. We compared balanced error rate (BER) against different SES levels measured by HOUsing-based SocioEconomic Status measure (HOUSES) index. As a possible mechanism for differential performance, we also compared incompleteness of EHR information relevant to asthma care by SES. Results: Asthmatic children with lower SES had larger BER than those with higher SES (eg, ratio = 1.35 for HOUSES Q1 vs Q2–Q4) and had a higher proportion of missing information relevant to asthma care (eg, 41% vs 24% for missing asthma severity and 12% vs 9.8% for undiagnosed asthma despite meeting asthma criteria). Discussion: Our study suggests that lower SES is associated with worse predictive model performance. It also highlights the potential role of incomplete EHR data in this differential performance and suggests a way to mitigate this bias. Conclusion: The HOUSES index allows AI researchers to assess bias in predictive model performance by SES. Although our case study was based on a small sample size and a single-site study, the study results highlight a potential strategy for identifying bias by using an innovative SES measure.