31 October 2019 in bioRxiv
Introduction: Artificial intelligence (AI) in healthcare has gained momentum with advances in affordable technology that has potential to help in diagnostics, predictive healthcare and personalized medicine. In pursuit of applying universal non-biased AI in healthcare, it is essential that data from different settings (gender, age and ethnicity) is represented. We present findings from beta-testing an AI-powered dermatological algorithm called Skin Image Search, by online dermatology company First Derm on Fitzpatrick 6 skin type (dark skin) dermatological conditions.Methods: 123 dermatological images selected from a total of 173 images retrospectively extracted from the electronic database of a Ugandan telehealth company, The Medical Concierge Group (TMCG) after getting their consent. Details of age, gender and dermatological clinical diagnosis were analyzed using R on R studio software to assess the diagnostic accuracy of the AI app along disease diagnosis and body part. Predictability levels of the AI app was graded on a scale of 0 to 5, where 0-no prediction made and 1-5 demonstrating reducing correct prediction.Results: 76 (62%) of the dermatological images were from females and 47 (38%) from males. The 5 most reported body parts were; genitals (20%), trunk (20%), lower limb (14.6%), face (12%) and upper limb (12%) with the AI app predicting a diagnosis in 62% of image body parts uploaded. Overall diagnostic accuracy of the AI app was low at 17% (21 out of 123 predictable images) with varying predictability levels correctness i.e. 1-8.9%, 2-2.4%, 3-2.4%, 4-1.6%, 5-1.6% with performance along individual diagnosis highest with dermatitis (80%).Conclusion: There is a need for diversity in the image datasets used when training dermatology algorithms for AI applications in clinical decision support as a means to increase accuracy and thus offer correct treatment across skin types and geographies.
31 August 2022 in JMIR Research Protocols
Background: Dermatological conditions are a relevant health problem. Each person has an average of 1.6 skin diseases per year, and consultations for skin pathology represent 20% of the total annual visits to primary care and around 35% are referred to a dermatology specialist. Machine learning (ML) models can be a good tool to help primary care professionals, as it can analyze and optimize complex sets of data. In addition, ML models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and classification. Objective: This study aims to perform a prospective validation of an image analysis ML model as a diagnostic decision support tool for the diagnosis of dermatological conditions. Methods: In this prospective study, 100 consecutive patients who visit a participant general practitioner (GP) with a skin problem in central Catalonia were recruited. Data collection was planned to last 7 months. Anonymized pictures of skin diseases were taken and introduced to the ML model interface (capable of screening for 44 different skin diseases), which returned the top 5 diagnoses by probability. The same image was also sent as a teledermatology consultation following the current stablished workflow. The GP, ML model, and dermatologist’s assessments will be compared to calculate the precision, sensitivity, specificity, and accuracy of the ML model. The results will be represented globally and individually for each skin disease class using a confusion matrix and one-versus-all methodology. The time taken to make the diagnosis will also be taken into consideration. Results: Patient recruitment began in June 2021 and lasted for 5 months. Currently, all patients have been recruited and the images have been shown to the GPs and dermatologists. The analysis of the results has already started. Conclusions: This study will provide information about ML models’ effectiveness and limitations. External testing is essential for regulating these diagnostic systems to deploy ML models in a primary care practice setting.
30 November 2009 in Journal of Telemedicine and Telecare
27 December 2022 in JMIR Preprints
BACKGROUND Dermatological conditions are a relevant health problem. Machine learning (ML) models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, especially for skin cancer detection and disease classification. OBJECTIVE The objective of this study was to perform a prospective validation of an image analysis ML model, which is capable of screening 44 skin diseases, comparing its diagnostic accuracy with that of General Practitioners (GPs) and teledermatology (TD) dermatologists in a real-life setting. METHODS Prospective, diagnostic accuracy study including 100 consecutive patients with a skin problem who visited a participating GP in central Catalonia, Spain, between June 2021 and October 2021. The skin issue was first assessed by the GPs. Then an anonymised skin disease picture was taken and uploaded to the ML application, which returned a list with the Top-5 possible diagnosis in order of probability. The same image was then sent to a dermatologist via TD for diagnosis, as per clinical practice. The GPs Top-3, ML model’s Top-5 and dermatologist’s Top-3 assessments were compared to calculate the accuracy, sensitivity, specificity and diagnostic accuracy of the ML models. RESULTS The overall Top-1 accuracy of the ML model (39%) was lower than that of GPs (64%) and dermatologists (72%). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained (n=82), the balanced Top-1 accuracy of the ML model increased (48%) and in the Top-3 (75%) was comparable to the GPs Top-3 accuracy (76%). The Top-5 accuracy of the ML model (89%) was comparable to the dermatologist Top-3 accuracy (90%). For the different diseases, the sensitivity of the model (Top-3 87% and Top-5 84%) is higher than that of the clinicians (Top-3 GPs 76% and Top-3 dermatologists 84%) only in the benign tumour pathology group, being on the other hand the most prevalent category (n=53). About the satisfaction of professionals, 92% of the GPs considered it as a usefuldiagnostic support tool (DST) for the differential diagnosis and in 60% of the cases as an aid in the final diagnosis of the skin lesion. CONCLUSIONS The overall diagnostic accuracy of the model in this study under real conditions is lower than that of both GPs and dermatologists, a fact that is consistent with the few existing prospective studies under real conditions. These results highlight the potential of the ML models to assist GPs as a DST for skin conditions especially in the differential diagnosis. However, external testing in real conditions is essential for data validation and regulating these AI diagnostic models, in order to deploy ML models in a Primary Care setting. CLINICALTRIAL The clinical trial has been approved by the IDIAP Jordi Gol i Gurina ethics committee with code 20-159P. Clinicaltrials.gov NCT04562168; https://clinicaltrials.gov/ct2/show/NCT04562168 (Archived by WebCite at http://www.webcitation.org/6V0tto0lZ). JMIR Res Protoc 2022;11(8):e37531 doi:10.2196/37531