We have attempted to reproduce the results in Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs, published in JAMA 2016; 316(22), using publicly available data sets. We re-implemented the main method in the original study since the source code is not available. The original study used non-public fundus images from EyePACS and three hospitals in India for training. We used a different EyePACS data set from Kaggle. The original study used the benchmark data set Messidor-2 to evaluate the algorithm’s performance. We used another distribution of the Messidor-2 data set, since the original data set is no longer available. In the original study, ophthalmologists re-graded all images for diabetic retinopathy, macular edema, and image gradability. We have one diabetic retinopathy grade per image for our data sets, and we assessed image gradability ourselves. We were not able to reproduce the original study’s results with publicly available data. Our algorithm’s area under the receiver operating characteristic curve (AUC) of 0.951 (95% CI, 0.947-0.956) on the Kaggle EyePACS test set and 0.853 (95% CI, 0.835-0.871) on Messidor-2 did not come close to the reported AUC of 0.99 on both test sets in the original study. This may be caused by the use of a single grade per image, or different data. This study shows the challenges of reproducing deep learning method results, and the need for more replication and reproduction studies to validate deep learning methods, especially for medical image analysis. Our source code and instructions are available at: https://github.com/mikevoets/jama16-retina-replication.

Increased levels of tumor-infiltrating lymphocytes (TILs) indicate favorable outcomes in many types of cancer. The manual quantification of immune cells is inaccurate and time-consuming for pathologists. Our aim is to leverage a computational solution to automatically quantify TILs in standard diagnostic hematoxylin and eosin-stained sections (H&E slides) from lung cancer patients. Our approach is to transfer an open-source machine learning method for the segmentation and classification of nuclei in H&E slides trained on public data to TIL quantification without manual labeling of the data. Our results show that the resulting TIL quantification correlates to the patient prognosis and compares favorably to the current state-of-the-art method for immune cell detection in non-small cell lung cancer (current standard CD8 cells in DAB-stained TMAs HR 0.34, 95% CI 0.17–0.68 vs. TILs in HE WSIs: HoVer-Net PanNuke Aug Model HR 0.30, 95% CI 0.15–0.60 and HoVer-Net MoNuSAC Aug model HR 0.27, 95% CI 0.14–0.53). Our approach bridges the gap between machine learning research, translational clinical research and clinical implementation. However, further validation is warranted before implementation in a clinical setting.

Foraminifera are single-celled marine organisms that construct shells that remain as fossils in the marine sediments. Classifying and counting these fossils are important in paleo-oceanographic and -climatological research. However, the identification and counting process has been performed manually since the 1800s and is laborious and time-consuming. In this work, we present a deep learning-based instance segmentation model for classifying, detecting, and segmenting microscopic foraminifera. Our model is based on the Mask R-CNN architecture, using model weight parameters that have learned on the COCO detection dataset. We use a fine-tuning approach to adapt the parameters on a novel object detection dataset of more than 7000 microscopic foraminifera and sediment grains. The model achieves a (COCO-style) average precision of $0.78$ on the classification and detection task, and $0.80$ on the segmentation task. When the model is evaluated without challenging sediment grain images, the average precision for both tasks increases to $0.84$ and $0.86$, respectively. Prediction results are analyzed both quantitatively and qualitatively and discussed. Based on our findings we propose several directions for future work and conclude that our proposed model is an important step towards automating the identification and counting of microscopic foraminifera.

Melanoma is the deadliest form of skin cancer, and early detection is crucial for patient survival. Computer systems can assist in melanoma detection, but are not widespread in clinical practice. In 2016, an open challenge in classification of dermoscopic images of skin lesions was announced. A training set of 900 images with corresponding class labels and semi-automatic/manual segmentation masks was released for the challenge. An independent test set of 379 images, of which 75 were of melanomas, was used to rank the participants. This article demonstrates the impact of ranking criteria, segmentation method and classifier, and highlights the clinical perspective. We compare five different measures for diagnostic accuracy by analysing the resulting ranking of the computer systems in the challenge. Choice of performance measure had great impact on the ranking. Systems that were ranked among the top three for one measure, dropped to the bottom half when changing performance measure. Nevus Doctor, a computer system previously developed by the authors, was used to participate in the challenge, and investigate the impact of segmentation and classifier. The diagnostic accuracy when using an automatic versus the semi-automatic/manual segmentation is investigated. The unexpected small impact of segmentation method suggests that improvements of the automatic segmentation method w.r.t. resemblance to semi-automatic/manual segmentation will not improve diagnostic accuracy substantially. A small set of similar classification algorithms are used to investigate the impact of classifier on the diagnostic accuracy. The variability in diagnostic accuracy for different classifier algorithms was larger than the variability for segmentation methods, and suggests a focus for future investigations. From a clinical perspective, the misclassification of a melanoma as benign has far greater cost than the misclassification of a benign lesion. For computer systems to have clinical impact, their performance should be ranked by a high-sensitivity measure.

Background: Increases in the proportion of the population with increased likelihood of cesarean section (CS) have been postulated as a driving force behind the rise in CS rates worldwide. The aim of the study was to assess if changes in selected maternal risk factors for CS are associated with changes in CS births from 1999 to 2016 in Norway. Methods and findings: This national population-based registry study utilizes data from 1,055,006 births registered in the Norwegian Medical Birth Registry from 1999 to 2016. The following maternal risk factors for CS were included: nulliparous/≥35 years, multiparous/≥35 years, pregestational diabetes, gestational diabetes, hypertensive disorders, previous CS, assisted reproductive technology, and multiple births. The proportion of CS births in 1999 was used to predict the number of CS births in 2016. The observed and predicted numbers of CS births were compared to determine the number of excess CS births, before and after considering the selected risk factors, for all births, and for births stratified by 0, 1, or >1 of the selected risk factors. The proportion of CS births increased from 12.9% to 16.1% (+24.8%) during the study period. The proportion of births with 1 selected risk factor increased from 21.3% to 26.3% (+23.5%), while the proportion with >1 risk factor increased from 4.5% to 8.8% (+95.6%). Stratification by the presence of selected risk factors reduced the number of excess CS births observed in 2016 compared to 1999 by 67.9%. Study limitations include lack of access to other important maternal risk factors and only comparing the first and the last year of the study period. Conclusions: In this study, we observed that after an initial increase, proportions of CS births remained stable from 2005 to 2016. Instead, both the size of the risk population and the mean number of risk factors per birth continued to increase. We observed a possible association between the increase in size of risk population and the additional CS births observed in 2016 compared to 1999. The increase in size of risk population and the stable CS rate from 2005 and onward may indicate consistent adherence to obstetric evidence-based practice in Norway.

Introduction The retropubic tension‐free vaginal tape procedure has been the preferred method for primary surgical treatment of stress and stress‐dominant mixed urinary incontinence in women for more than 20 years. In this study, we assessed associations between surgeon’s experience with the primary tension‐free vaginal tape procedure and both perioperative complications and recurrence rates. Material and methods Using a consecutive case‐series design, we assessed 596 patients treated with primary retropubic tension‐free vaginal tape surgery performed by 18 surgeons from 1998 through 2012, with follow‐up through 2015 (maximum follow‐up time: 10 years per patient). Data on perioperative complications and recurrence of stress urinary incontinence from medical records was transferred to a case report form. Surgeon’s experience with the tension‐free vaginal tape procedure was defined as number of such procedures performed as lead surgeon (1‐19 (“beginners”), 20‐49, and ≥50 procedures). All analyses were done with a 5% level of statistical significance. We applied the Chi‐squared test in the assessment of perioperative complications. The regression analyses of recurrence rate by number of tension‐free vaginal tape procedures performed were restricted to the three surgeons who performed ≥50 procedures. Results We found a significantly higher rate of bladder perforations (p = 0.03) and a higher rate of urinary retentions among patients whose tension‐free vaginal tape procedures were performed by “beginners” (p = 0.06). We observed a significant reduction in recurrence rates with increasing number of tension‐free vaginal tape procedures for one surgeon (p = 0.03). Conclusions Surgeon’s experience with the tension‐free vaginal tape procedure is associated with the risk of bladder perforation and urinary retention, and may be associated with the long‐term effectiveness of the procedure.