The OperA (Optimising Colorectal Cancer Prevention through Personalized Treatment with Artificial Intelligence) project aims to transform colorectal cancer care through artificial intelligence (AI) innovations. Recognizing that legal and ethical challenges remain key obstacles to clinical integration, this Delphi study sought to identify and prioritize such concerns in the context of gastrointestinal (GI) endoscopy. Fourteen international experts participated in a 2-round Delphi process. In round 1, the steering committee, with feedback from participants, proposed legal and ethical issues pertaining to AI in endoscopy. Round 2 involved iterative rating and refinement of these issues to achieve consensus on their importance. Consensus was reached on 10 key statements spanning 3 thematic domains: data governance, medicolegal implications, and equity and bias. Experts emphasized the need for robust data protection, transparent algorithmic development, and institutional clarity on data ownership. Liability concerns related to AI-assisted diagnosis and automated reporting were highlighted, alongside calls for guidance from legal and professional bodies. Finally, participants underscored the importance of demographic diversity in training data sets and transparent reporting practices to mitigate bias and ensure equitable AI deployment. As AI tools become increasingly integrated into the clinical practice of gastroenterology, addressing legal, ethical, and equity-related challenges is essential. This expert consensus provides a foundation for developing guidelines and regulatory frameworks to support responsible AI adoption in GI endoscopy.
Background: The Mayo endoscopic subscore (MES) is an important quantitative measure of disease activity in ulcerative colitis. Colonoscopy reports in routine clinical care usually characterize ulcerative colitis disease activity using free text description, limiting their utility for clinical research and quality improvement. We sought to develop algorithms to classify colonoscopy reports according to their MES. Methods: We annotated 500 colonoscopy reports from 2 health systems. We trained and evaluated 4 classes of algorithms. Our primary outcome was accuracy in identifying scorable reports (binary) and assigning an MES (ordinal). Secondary outcomes included learning efficiency, generalizability, and fairness. Results: Automated machine learning models achieved 98% and 97% accuracy on the binary and ordinal prediction tasks, outperforming other models. Binary models trained on the University of California, San Francisco data alone maintained accuracy (96%) on validation data from Zuckerberg San Francisco General. When using 80% of the training data, models remained accurate for the binary task (97% [n = 320]) but lost accuracy on the ordinal task (67% [n = 194]). We found no evidence of bias by gender (P = .65) or area deprivation index (P = .80). Conclusions: We derived a highly accurate pair of models capable of classifying reports by their MES and recognizing when to abstain from prediction. Our models were generalizable on outside institution validation. There was no evidence of algorithmic bias. Our methods have the potential to enable retrospective studies of treatment effectiveness, prospective identification of patients meeting study criteria, and quality improvement efforts in inflammatory bowel diseases.
Background and study aims There is limited consensus on the optimal method for measuring disease severity in familial adenomatous polyposis (FAP). We aimed to systematically review the operating properties of existing endoscopic severity indices for FAP. Methods We searched MEDLINE, EMBASE, and the Cochrane Library from inception to February 2023 to identify randomized controlled trials (RCTs) that utilized endoscopic outcomes or studies that evaluated the operating properties of endoscopic disease severity indices in FAP. Results A total of 134 studies were included. We evaluated scoring indices and component items of scoring indices, such as polyp count, polyp size, and histology. Partial validation was observed for polyp count and size. The most commonly reported scoring index was the Spigelman classification system, which was used for assessing the severity of duodenal involvement. A single study reported almost perfect interobserver and intra-observer agreement for this system. The InSIGHT polyposis staging system, which was used for assessing colorectal polyp burden, has been partially validated. It showed substantial interobserver reliability; however, the intra-observer reliability was not assessed. Novel criteria for high-risk gastric polyps have been developed and assessed for interobserver reliability. However, these criteria showed a poor level of agreement. Other scoring indices assessing the anal transition zone, duodenal, and colorectal polyps have not undergone validation. Conclusions There are no fully validated endoscopic disease severity indices for FAP. Development and validation of a reliable and responsive endoscopic disease severity instrument will be informative for clinical care and RCTs of pharmacological therapies for FAP.
Importance: Electronic health records (EHR) data are growing in importance as a source of evidence on real-world treatment effects. However, many clinical important measures are not directly captured as structured data by these systems, limiting their utility for research and quality improvement. Although this information can usually be manually abstracted from clinical notes, this process is expensive and subject to variability. Natural language processing (NLP) is a scalable alternative but has historically been subject to multiple limitations including insufficient accuracy, data hunger, technical complexity, poor generalizability, algorithmic unfairness, and an outsized carbon footprint. Objective: Compare different algorithmic approaches for classifying colonoscopy reports according to their ulcerative colitis Mayo endoscopic subscores Design: Other observational study – NLP algorithm development and validation Setting: Academic medical center (UCSF) and safety-net hospital (ZSFG) in California Participants: Patients with ulcerative colitis Exposures: Colonoscopy Main Outcomes and Measures: The primary outcome was accuracy in identifying reports suitable for Mayo subscoring (binary yes/no) and then separately assigning a Mayo subscore where relevant (ordinal). Secondary outcomes included learning efficiency from training data, generalizability, computational costs, fairness, and sustainability. Results: Using automated machine learning (autoML) we trained a pair of classifiers that were 98% [91-99%] accurate at determining which reports to score and 97% [88-99%] accurate at assigning the correct Mayo endoscopic subscore. The binary classifiers trained on UCSF data achieved 96% accuracy on hold-out test data from ZSFG. Training these classifiers required 4 hours of computation on a standard laptop. Classification errors were not associated with either gender or area deprivation index. The carbon footprint of this approach was 24x less than current deep learning algorithms for clinical text classification. Conclusions and Relevance: We identified autoML as an efficient and robust method for training clinical text classifiers. AutoML-trained classifiers demonstrated many favorable properties including generalizability, limited effort needed for data annotation and algorithm training, fairness, and sustainability. More generally, these results support the feasibility of using unstructured EHR data to generate real-world evidence and drive continuous improvements in learning health systems. Key Points: Question: Is natural language processing (NLP) a viable alternative to manually abstracting disease activity from procedure notes? Findings: We compared different methods for abstracting the ulcerative colitis Mayo endoscopic subscore from colonoscopy reports. Classifiers trained using automated machine learning (autoML) achieved the greatest accuracy (97%), recognized when to abstain, generalized well to other health systems, required limited effort for annotation and programming, demonstrated fairness, and had a small carbon footprint. Meaning: NLP methods like autoML appear to be sufficiently mature technologies for clinical text classification, and thus are poised to enable many downstream endeavors using electronic health records data.