Artificial Intelligence (AI) systems could improve system efficiency by supporting clinicians in making appropriate referrals. However, they are imperfect by nature and misdiagnoses, if not correctly identified, can have consequences for patient care. In this paper, findings from an online survey are presented to understand the aptitude of GPs (n = 50) in appropriately trusting or not trusting the output of a fictitious AI-based decision support tool when assessing skin lesions, and to identify which individual characteristics could make GPs less prone to adhere to erroneous diagnostics results. The findings suggest that, when the AI was correct, the GPs’ ability to correctly diagnose a skin lesion significantly improved after receiving correct AI information, from 73.6% to 86.8% (X2 (1, N = 50) = 21.787, p < 0.001), with significant effects for both the benign (X2 (1, N = 50) = 21, p < 0.001) and malignant cases (X2 (1, N = 50) = 4.654, p = 0.031). However, when the AI provided erroneous information, only 10% of the GPs were able to correctly disagree with the indication of the AI in terms of diagnosis (d-AIW M: 0.12, SD: 0.37), and only 14% of participants were able to correctly decide the management plan despite the AI insights (d-AIW M:0.12, SD: 0.32). The analysis of the difference between groups in terms of individual characteristics suggested that GPs with domain knowledge in dermatology were better at rejecting the wrong insights from AI.
Introduction: Care homes have been severely affected by the SARS-CoV-2 pandemic. Rapid antigen testing could identify most SARS-CoV-2 infected staff and visitors before they enter homes. We explored implementation of staff and visitor testing protocols using lateral flow devices (LFDs). Methods: An evaluation of a SARS-CoV-2 LFD-based testing protocol in 11 care homes in Liverpool, UK, including staff and visitor testing, plus a qualitative exploratory study in nine of these homes. The proportion of pilot homes with outbreaks, and outbreak size, were compared to non-pilot homes in Liverpool. Adherence to testing protocols was evaluated. Fifteen staff were interviewed, and transcript data were thematically coded using an iterative analysis to identify and categorize factors influencing testing implementation. Results: In total, 1,638 LFD rapid tests were performed on 407 staff. Protocol adherence was poor with 8.6% of staff achieving >75% protocol adherence, and 25.3% achieving $\ge$50%. Six care homes had outbreaks during the study. Compared to non-pilot care homes, there was no evidence of significant difference in the proportion of homes with outbreaks, or the size of outbreaks. Qualitative data showed difficulty implementing testing strategies due to excessive work burden. Factors influencing adherence related to test integration and procedural factors, socio-economic factors, cognitive overload and the emotional value of testing. Conclusion: Implementation of staff and visitor care home LFD testing protocols was poorly adhered to and consequently did not reduce the number or scale of COVID-19 outbreaks. More focus is needed on the contextual and behavioural factors that influence protocol adherence.
Introduction: Care home residents are at high risk of dying from COVID-19. Regular testing, producing rapid and reliable results is important in this population because infections spread quickly, and presentations are often atypical or asymptomatic. This study evaluated current testing pathways in care homes to explore the role of point-of-care tests (POCTs). Methods: Ten staff from eight care homes, purposively sampled to reflect care organisational attributes that influence outbreak severity, underwent a semi-structured remote videoconference interview. Transcripts were analysed using process mapping tools and framework analysis focussing on perceptions about, gaps within, and needs arising from, current pathways. Results: Four main steps were identified in testing: infection prevention, preparatory steps, swabbing procedure, and management of residents. Infection prevention was particularly challenging for mobile residents with cognitive impairment. Swabbing and preparatory steps were resource-intensive, requiring additional staff resource. Swabbing required flexibility and staff who were familiar to the resident. Frequent approaches to residents were needed to ensure they would participate at a suitable time. After-test management varied between sites. Several homes reported deviating from government guidance to take more cautious approaches, which they perceived to be more robust. Conclusion: Swab-based testing is organisationally complex and resource-intensive in care homes. It needs to be flexible to meet the needs of residents and provide care homes with rapid information to support care decisions. POCT could help address gaps but the complexity of the setting means that each technology must be evaluated in context before widespread adoption in care homes.
20 November 2021 in Journal of Patient Safety and Risk Management
Introduction: Successful adoption of POCTs (Point-of-Care tests) for COVID-19 in care homes requires the identification of ideal use cases and a full understanding of the contextual and usability factors that affect test results and minimise biosafety risks. This paper presents a scoping-usability and test performance study of a microfluidic immunofluorescence assay for COVID-19 in care homes. Methods: A mixed-methods evaluation was conducted in four UK care homes to scope usability and to assess the agreement with qRT-PCR. A dry run with luminescent dye was conducted to explore biosafety issues. Results: The agreement analysis was conducted on 227 asymptomatic participants (159 staff and 68 residents) and 14 symptomatic participants (5 staff and 9 residents). Asymptomatic specimens showed 50% (95% CI:1.3%−98.7%) positive agreement and 96% (95% CI: 92.5%−98.1%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.911 (95% CI: 0.857−0.965). Symptomatic specimens showed 83.3% (95% CI: 35.9%−99.6%) positive agreement and 100% (95% CI: 63.1%−100%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.857 (95% CI: 0.549−1). The dry run highlighted four main sources of contamination that led to the modification of the standard operating procedures. Simulation post-modification showed no further evidence of contamination. Conclusion: Careful consideration of biosafety issues and contextual factors associated with care home are mandatory for safe use the POCT. Whilst POCT may have some utility for ruling out COVID-19, further diagnostic accuracy evaluations are needed to promote effective adoption.
Introduction Successful adoption of POCTs (Point-of-Care tests) for COVID-19 in care homes requires the identification of ideal use cases and a full understanding of contextual and usability factors that affect test results and minimise biosafety risks. This paper presents findings from a scoping-usability and test performance study of a microfluidic immunofluorescence assay for COVID-19 in care homes. Methods A mixed-methods evaluation was conducted in four UK care homes to scope usability and to assess the agreement with qRT-PCR. A dry run with luminescent dye was carried out to explore biosafety issues. Results The agreement analysis was carried out on 227 asymptomatic participants (159 staff and 68 residents) and 14 symptomatic participants (5 staff and 9 residents). Asymptomatic specimens showed 50% (95% CI: 1.3%-98.7%) positive agreement and 96% (95% CI: 92.5%-98.1%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.911 (95% CI: 0.857-0.965). Symptomatic specimens showed 83.3% (95% CI: 35.9%-99.6%) positive agreement and 100% (95% CI: 63.1%-100%) negative agreement with overall prevalence and bias-adjusted Kappa (PABAK) of 0.857 (95% CI: 0.549-1). The dry run showed four main sources of contamination that led to the modification of the standard operating procedures. Simulation after modification showed no further evidence of contamination. Conclusion Careful consideration of biosafety issues and contextual factors associated with care home are mandatory for safe use the POCT. Whilst POCT may have some utility for ruling out COVID-19, further diagnostic accuracy evaluations are needed to promote effective adoption.
Introduction Care home residents are at high risk of dying from COVID-19. Regular testing producing rapid and reliable results is important in this population because infections spread quickly and presentations are often atypical or asymptomatic. This study evaluated current testing pathways in care homes to explore the role of point-of-care tests (POCTs). Methods Ten staff from eight care homes, purposively sampled to reflect care organisational attributes that influence outbreak severity, underwent a semi-structured remote videoconference interview. Transcripts were analysed using process mapping tools and framework analysis focussing on perceptions about, gaps within, and needs arising from, current pathways. Results Four main steps were identified in testing: infection prevention, preparatory steps, swabbing procedure, and management of residents. Infection prevention was particularly challenging for mobile residents with cognitive impairment. Swabbing and preparatory steps were resource-intensive, requiring additional staff resource. Swabbing required flexibility and staff who were familiar to the resident. Frequent approaches to residents were needed to ensure they would participate at a suitable time. After-test management varied between sites. Several homes reported deviating from government guidance to take more cautious approaches, which they perceived to be more robust. Conclusion Swab-based testing is organisationally complex and resource-intensive in care homes. It needs to be flexible to meet the needs of residents and provide care homes with rapid information to support care decisions. POCT could help address gaps but the complexity of the setting means that each technology must be evaluated in context before widespread adoption in care homes. Key-points Testing for COVID-19 in care homes is complex and requires reconfiguration of staffing and environment. Isolation and testing procedures are challenged when providing person-centred care to people with dementia. Point-of-care testing results could give care homes greater flexibility to test in person-centred ways. There was evidence that care home staff interpret testing guidance, rather than follow it verbatim. Each POCT must be evaluated in the context of care homes to understand its effect on care home processes.