Along with recent developments in deep learning techniques, computer-aided diagnosis (CAD) has been growing rapidly in the medical imaging field. In this work, we evaluate the deep learning-based CAD algorithm (DCAD) for detecting and localizing 3 major thoracic abnormalities visible on chest radiographs (CR) and to compare the performance of physicians with and without the assistance of the algorithm. A subset of 244 subjects (60% abnormal CRs) was evaluated. Abnormal findings included mass/nodules (55%), consolidation (21%), and pneumothorax (24%). Observer performance tests were conducted to assess whether the performance of physicians could be enhanced with the algorithm. The area under the receiver operating characteristic (ROC) curve (AUC) and the area under the jackknife alternative free-response ROC (JAFROC) were measured to evaluate the performance of the algorithm and physicians in image classification and lesion detection, respectively. The AUCs for nodule/mass, consolidation, and pneumothorax were 0.9883, 1.000, and 0.9997, respectively. For the image classification, the overall AUC of the pooled physicians was 0.8679 without DCAD and 0.9112 with DCAD. Regarding lesion detection, the pooled observers exhibited a weighted JAFROC figure of merit (FOM) of 0.8426 without DCAD and 0.9112 with DCAD. DCAD for CRs could enhance physicians’ performance in the detection of 3 major thoracic abnormalities.

The common paradigm of CNN-based action recognition modelsis to simply use the average of the dense predictions from every frame. However, these dense predictions are inefficient since all frames are evenly utilized regardless of the existence of the action. In real-time action recognition applications where the input video is untrimmed, the dense prediction is even more severely inefficient. Instead of these dense predictions, we propose a lightweight CNN-based sampler that selects frames that are relevant to the action in the video. The proposed sampler is trained by a self-supervised learning method that mixes one irrelevant frame in an input video clip and finds that irrelevant frame. With the trained sampler, we select frames to be used as inputs to the action recognition model. Our sampler can be combined with any action recognition models and works independently of them. Through various experiments, we demonstrate that the proposed sampler is not only time and memory-efficient but also significantly increases performance in the action recognition benchmarks, including HMDB51 and UCF101. Finally, we applied the self-supervised sampler to the real-time surveillance system in the subway station, and improved the action recognition performance by 7% with 80% reduction in computation. Code and dataset are available in https://github.com/seominseok0429/SelSupSampler and https://aihub.or.kr/aidata/34124 .

Deep learning-based methods have achieved good performance in various recognition benchmarks mostly by utilizing single modalities. As different modalities contain complementary information to each other, multi-modal based methods are proposed to implicitly utilize them. In this paper, we propose a simple technique, called correspondence learning (CL), which explicitly learns the relationship among multiple modalities. The multiple modalities in the data samples are randomly mixed among different samples. If the modalities are from the same sample (not mixed), then they have positive correspondence, and vice versa. CL is an auxiliary task for the model to predict the correspondence among modalities. The model is expected to extract information from each modality to check correspondence and achieve better representations in multi-modal recognition tasks. In this work, we first validate the proposed method in various multi-modal benchmarks including CMU Multimodal Opinion-Level Sentiment Intensity (CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) sentiment analysis datasets. In addition, we propose a fraud detection method using the learned correspondence among modalities. To validate this additional usage, we collect a multi-modal dataset for fraud detection using real-world samples for reverse vending machines.

EGFR mutation screening in non–small cell lung cancer (NSCLC) remains variable globally and represents a significant care gap despite international recommendations and molecular testing guidelines. Recently, the use of deep learning (DL) methods to extract clinically actionable features from routine histology images has gained regulatory approval for multiple clinical applications. Therefore, the integration of predictive DL to complement molecular EGFR mutation screening may improve biomarker testing rates in NSCLC. To address this unmet need, we developed and validated Lunit SCOPE Genotype Predictor, a DL model trained and tuned using more than 12,000 whole-slide images, that predicts EGFR mutation status from routine hematoxylin and eosin images. Using a diverse dataset (n = 1,461) that captures histologic subtypes, multiple whole-slide scanners, different scan magnifications, and specimen types, we report an overall area under the ROC curve (AUROC) of 0.905. The model demonstrates robust performance across specimen types (biopsies and surgical resections, 0.804 and 0.912, respectively), histologic subtypes (adenocarcinoma and non-adenocarcinoma, 0.880 and 0.801, respectively), and EGFR mutation subtypes (AUROC, 0.854–0.931). Additionally, across a second independent test set (n = 599) sourced from 11 countries utilizing five different slide scanners, Lunit SCOPE Genotype Predictor achieved a robust AUROC of 0.860. Furthermore, across a multi-scanner test set (n = 2,261), EGFR mutation predictions were concordant in 90.4% of cases among five of six frequently used slide scanners. This validation across diverse clinical settings represents a vital step toward the application of artificial intelligence–based digital pathology tools in routine clinical practice to augment molecular EGFR mutation screening. Significance: DL predicts EGFR mutations in NSCLC from routine histology images, achieving an overall AUROC of 0.905 and 0.860 in two independent test sets across histologic subtypes, mutation subtypes, and imaging platforms.