Multiple studies have compared the performance of artificial intelligence (AI)-based models for automated skin cancer classification to human experts, thus setting the cornerstone for a successful translation of AI-based tools into clinicopathological practice. The objective of the study was to systematically analyse the current state of research on reader studies involving melanoma and to assess their potential clinical relevance by evaluating three main aspects: test set characteristics (holdout/out-of-distribution data set, composition), test setting (experimental/clinical, inclusion of metadata) and representativeness of participating clinicians. PubMed, Medline and ScienceDirect were screened for peer-reviewed studies published between 2017 and 2021 and dealing with AI-based skin cancer classification involving melanoma. The search terms skin cancer classification, deep learning, convolutional neural network (CNN), melanoma (detection), digital biomarkers, histopathology and whole slide imaging were combined. Based on the search results, only studies that considered direct comparison of AI results with clinicians and had a diagnostic classification as their main objective were included. A total of 19 reader studies fulfilled the inclusion criteria. Of these, 11 CNN-based approaches addressed the classification of dermoscopic images; 6 concentrated on the classification of clinical images, whereas 2 dermatopathological studies utilised digitised histopathological whole slide images. All 19 included studies demonstrated superior or at least equivalent performance of CNN-based classifiers compared with clinicians. However, almost all studies were conducted in highly artificial settings based exclusively on single images of the suspicious lesions. Moreover, test sets mainly consisted of holdout images and did not represent the full range of patient populations and melanoma subtypes encountered in clinical practice.

Due to their ability to solve complex problems, deep neural networks (DNNs) are becoming increasingly popular in medical applications. However, decision-making by such algorithms is essentially a black-box process that renders it difficult for physicians to judge whether the decisions are reliable. The use of explainable artificial intelligence (XAI) is often suggested as a solution to this problem. We investigate how XAI is used for skin cancer detection: how is it used during the development of new DNNs? What kinds of visualisations are commonly used? Are there systematic evaluations of XAI with dermatologists or dermatopathologists? Google Scholar, PubMed, IEEE Explore, Science Direct and Scopus were searched for peer-reviewed studies published between January 2017 and October 2021 applying XAI to dermatological images: the search terms histopathological image, whole-slide image, clinical image, dermoscopic image, skin, dermatology, explainable, interpretable and XAI were used in various combinations. Only studies concerned with skin cancer were included. 37 publications fulfilled our inclusion criteria. Most studies (19/37) simply applied existing XAI methods to their classifier to interpret its decision-making. Some studies (4/37) proposed new XAI methods or improved upon existing techniques. 14/37 studies addressed specific questions such as bias detection and impact of XAI on man-machine-interactions. However, only three of them evaluated the performance and confidence of humans using CAD systems with XAI. XAI is commonly applied during the development of DNNs for skin cancer detection. However, a systematic and rigorous evaluation of its usefulness in this scenario is lacking.

Clinicians and pathologists traditionally use patient data in addition to clinical examination to support their diagnoses. We investigated whether a combination of histologic whole slides image (WSI) analysis based on convolutional neural networks (CNNs) and commonly available patient data (age, sex and anatomical site of the lesion) in a binary melanoma/nevus classification task could increase the performance compared with CNNs alone. We used 431 WSIs from two different laboratories and analysed the performance of classifiers that used the image or patient data individually or three common fusion techniques. Furthermore, we tested a naive combination of patient data and an image classifier: for cases interpreted as 'uncertain' (CNN output score <0.7), the decision of the CNN was replaced by the decision of the patient data classifier. The CNN on its own achieved the best performance (mean ± standard deviation of five individual runs) with AUROC of 92.30% ± 0.23% and balanced accuracy of 83.17% ± 0.38%. While the classification performance was not significantly improved in general by any of the tested fusions, naive strategy of replacing the image classifier with the patient data classifier on slides with low output scores improved balanced accuracy to 86.72% ± 0.36%. In most cases, the CNN on its own was so accurate that patient data integration did not provide any benefit. However, incorporating patient data for lesions that were classified by the CNN with low 'confidence' improved balanced accuracy.

Background: Mucosal and acral melanoma respond worse to immune checkpoint inhibitors (ICI) than cutaneous melanoma. MDM2/4 as well as EGFR amplifications are supposed to be associated with hyperprogression on ICI in diverse cancers. We therefore investigated the response of metastatic acral and mucosal melanoma to ICI in regard to MDM2/4 or EGFR amplifications and melanoma type. Methods: We conducted a query of our melanoma registry, looking for patients with metastatic acral or mucosal melanoma treated by ICI. Whole exome sequencing, FISH and immunohistochemistry on melanoma tissue could be performed on 45 of the total cohort of 51 patients. Data were correlated with patients’ responses to ICI and survival. Results: 22 out of 51 patients had hyperprogressive disease (an increase in tumor load of >50% at the first staging). Hyperprogression occurred more often in case of MDM2/4 or EGFR amplification or <1% PD-L1 positive tumor cells. Nevertheless, this association was not significant. Interestingly, the anorectal melanoma type and the presence of liver metastases were significantly associated with worse survival. Conclusions: So far, we found no reliable predictive marker for patients who develop hyperprogression on ICI, specifically with regard to MDM2/4 or EGFR amplifications. Nevertheless, patients with anorectal melanoma, liver metastases or melanoma with amplified MYC seem to have an increased risk of not benefitting from ICI.

Immune checkpoint inhibitors (ICIs) belong to the therapeutic armamentarium in advanced hepatocellular carcinoma (HCC). However, only a minority of patients benefit from immunotherapy. Therefore, we aimed to identify indicators of therapy response. This multicenter analysis included 99 HCC patients. Progression-free (PFS) and overall survival (OS) were studied by Kaplan-Meier analyses for clinical parameters using weighted log-rank testing. Next-generation sequencing (NGS) was performed in a subset of 15 patients. The objective response (OR) rate was 19% median OS (mOS)16.7 months. Forty-one percent reached a PFS > 6 months; these patients had a significantly longer mOS (32.0 vs. 8.5 months). Child-Pugh (CP) A and B patients showed a mOS of 22.1 and 12.1 months, respectively. Ten of thirty CP-B patients reached PFS > 6 months, including 3 patients with an OR. Tumor mutational burden (TMB) could not predict responders. Of note, antibiotic treatment within 30 days around ICI initiation was associated with significantly shorter mOS (8.5 vs. 17.4 months). Taken together, this study shows favorable outcomes for OS with low AFP, OR, and PFS > 6 months. No specific genetic pattern, including TMB, could identify responders. Antibiotics around treatment initiation were associated with worse outcome, suggesting an influence of the host microbiome on therapy success.

One prominent application for deep learning-based classifiers is skin cancer classification on dermoscopic images. However, classifier evaluation is often limited to holdout data which can mask common shortcomings such as susceptibility to confounding factors. To increase clinical applicability, it is necessary to thoroughly evaluate such classifiers on out-of-distribution (OOD) data. The objective of the study was to establish a dermoscopic skin cancer benchmark in which classifier robustness to OOD data can be measured. Using a proprietary dermoscopic image database and a set of image transformations, we create an OOD robustness benchmark and evaluate the robustness of four different convolutional neural network (CNN) architectures on it. The benchmark contains three data sets-Skin Archive Munich (SAM), SAM-corrupted (SAM-C) and SAM-perturbed (SAM-P)-and is publicly available for download. To maintain the benchmark's OOD status, ground truth labels are not provided and test results should be sent to us for assessment. The SAM data set contains 319 unmodified and biopsy-verified dermoscopic melanoma (n = 194) and nevus (n = 125) images. SAM-C and SAM-P contain images from SAM which were artificially modified to test a classifier against low-quality inputs and to measure its prediction stability over small image changes, respectively. All four CNNs showed susceptibility to corruptions and perturbations. This benchmark provides three data sets which allow for OOD testing of binary skin cancer classifiers. Our classifier performance confirms the shortcomings of CNNs and provides a frame of reference. Altogether, this benchmark should facilitate a more thorough evaluation process and thereby enable the development of more robust skin cancer classifiers.