Computational histopathology is focused on the automatic analysis of rich phenotypic information contained in gigabyte whole slide images, aiming at providing cancer patients with more accurate diagnosis, prognosis, and treatment recommendations. Nowadays deep learning is the mainstream methodological choice in computational histopathology. Transformer, as the latest technological advance in deep learning, learns feature representations and global dependencies based on self-attention mechanisms, which is increasingly gaining prevalence in this field. This paper presents a comprehensive review of state-of-the-art vision transformers that have been explored in histopathological image analysis for classification, segmentation, and survival risk regression applications. We first overview preliminary concepts and components built into vision transformers. Various recent applications including whole slide image classification, histological tissue component segmentation, and survival outcome prediction with tailored transformer architectures are then discussed. We finally discuss key challenges revolving around the use of vision transformers and envisioned future perspectives. We hope that this review could provide an elaborate guideline for readers to explore vision transformers in computational histopathology, such that more advanced techniques assisting in the precise diagnosis and treatment of cancer patients could be developed.

This study aimed to explore the prognostic impact of spatial distribution of tumor‐infiltrating lymphocytes (TILs) quantified by deep learning (DL) approaches based on digitalized whole‐slide images stained with hematoxylin and eosin in patients with colorectal cancer (CRC). The prognostic impact of spatial distributions of TILs in patients with CRC was explored in the Yonsei cohort (n = 180) and validated in The Cancer Genome Atlas (TCGA) cohort (n = 268). Two experienced pathologists manually measured TILs at the most invasive margin (IM) as 0–3 by the Klintrup–Mäkinen (KM) grading method and this was compared to DL approaches. Inter‐rater agreement for TILs was measured using Cohen's kappa coefficient. On multivariate analysis of spatial TIL features derived by DL approaches and clinicopathological variables including tumor stage, microsatellite instability, and KRAS mutation, TIL densities within 200 μm of the IM (f_im200) remained the most significant prognostic factor for progression‐free survival (PFS) (hazard ratio [HR] 0.004 [95% confidence interval, CI, 0.0001–0.15], p = 0.0028) in the Yonsei cohort. On multivariate analysis using the TCGA dataset, f_im200 retained prognostic significance for PFS (HR 0.031 [95% CI 0.001–0.645], p = 0.024). Inter‐rater agreement of manual KM grading was insignificant in the Yonsei (κ = 0.109) and the TCGA (κ = 0.121) cohorts. The survival analysis based on KM grading showed statistically significant different PFS in the TCGA cohort, but not the Yonsei cohort. Automatic quantification of TILs at the IM based on DL approaches shows prognostic utility to predict PFS, and could provide robust and reproducible TIL density measurement in patients with CRC.

Histological Gleason grading of tumor patterns is one of the most powerful prognostic predictors in prostate cancer. However, manual analysis and grading performed by pathologists are typically subjective and time-consuming. In this paper, we present an automatic technique for Gleason grading of prostate cancer from H&E stained whole slide pathology images using a set of novel completed and statistical local binary pattern (CSLBP) descriptors. First the technique divides the whole slide image (WSI) into a set of small image tiles, where salient tumor tiles with high nuclei densities are selected for analysis. The CSLBP texture features that encode pixel intensity variations from circularly surrounding neighborhoods are extracted from salient image tiles to characterize different Gleason patterns. Finally, the CSLBP texture features computed from all tiles are integrated and utilized by the multi-class support vector machine (SVM) that assigns patient slides with different Gleason scores such as 6, 7 or >=8. Experiments have been performed on 312 different patient cases selected from the cancer genome atlas (TCGA) and have achieved superior performances over state-of-the-art texture descriptors and baseline methods including deep learning models for prostate cancer Gleason grading.

High tumor mutation burden (TMB-H) could result in an increased number of neoepitopes from somatic mutations expressed by a patient’s own tumor cell which can be recognized and targeted by neighboring tumor-infiltrating lymphocytes (TILs). Deeper understanding of spatial heterogeneity and organization of tumor cells and their neighboring immune infiltrates within tumors could provide new insights into tumor progression and treatment response. Here we first developed computational approaches using whole slide images (WSIs) to predict bladder cancer patients’ TMB status and TILs across tumor regions, and then investigate spatial heterogeneity and organization of regions harboring TMB-H tumor cells and TILs within tumors, as well as their prognostic utility. Results: In experiments using WSIs from The Cancer Genome Atlas (TCGA) bladder cancer (BLCA), our findings show that computational pathology can reliably predict patient-level TMB status and delineate spatial TMB heterogeneity and co-organization with TILs. TMB-H patients with low spatial heterogeneity enriched with high TILs show improved overall survival. Computational approaches using WSIs have the potential to provide rapid and cost-effective TMB testing and TILs detection. Survival analysis illuminates potential clinical utility of spatial heterogeneity and co-organization of TMB and TILs as a prognostic biomarker in BLCA which warrants further validation in future studies.

For real-world automatic sleep-stage classification tasks, various existing deep learning-based models are biased toward the majority with a high proportion. Because of the unique sleep structure, most of the current polysomnography (PSG) datasets suffer an inherent class imbalance problem (CIP), in which the number of each sleep stage is severely unequal. In this study, we first define the class imbalance factor (CIF) to describe the level of CIP quantitatively. Afterward, we propose two balancing methods to alleviate this problem from the dataset quantity and the relationship between the class distribution and the applied model, respectively. The first one is to employ the data augmentation (DA) with the generative adversarial network (GAN) model and different intensities of Gaussian white noise (GWN) to balance samples, thereinto, GWN addition is specifically tailored to deep learning-based models, which can work on raw electroencephalogram (EEG) data while preserving their properties. In addition, we try to balance the relationship between the imbalanced class and biased network model to achieve a balanced state with the help of class distribution and neuroscience principles. We further propose an effective deep convolutional neural network (CNN) model utilizing bidirectional long short-term memory (Bi-LSTM) with single-channel EEG as the baseline. It is used for evaluating the efficiency of two balancing approaches on three imbalanced PSG datasets (CCSHS, Sleep-EDF, and Sleep-EDF-V1). The qualitative and quantitative evaluation of experimental results demonstrates that the proposed methods could not only show the superiority of class balancing through the confusion matrix and classwise metrics, but also get better N1 stage and whole stages classification accuracies compared to other state-of-the-art approaches.