Multiple instance learning (MIL), a powerful strategy for weakly supervised learning, is able to perform various prediction tasks on gigapixel whole slide images (WSIs). However, the tens of thousands of patches in WSIs usually incur a vast computational burden for image augmentation, limiting the performance improvement in MIL. Currently, the feature augmentation-based MIL framework is a promising solution, while existing methods such as mixup often produce unrealistic features. To explore a more efficient and practical augmentation method, we introduce the diffusion model (DM) into MIL for the first time and propose a feature augmentation framework called AugDiff. The diverse generation capabilities of DM guarantee a various range of feature augmentations, while its iterative generation approach effectively preserves semantic integrity during these augmentations. We conduct extensive experiments over four distinct cancer datasets, two different feature extractors, and three prevalent MIL algorithms to evaluate the performance of AugDiff. Ablation study and visualization further verify the effectiveness. Moreover, we highlight AugDiff’s higher-quality augmented feature over image augmentation and its superiority over self-supervised learning. The generalization over external datasets indicates its broader applications. The code is open-sourced on https://github.com/szc19990412/AugDiff .

In histopathology, the tissue slides are usually stained by common H&E stain or special stains (MAS, PAS, and PASM, etc.) to clearly show specific tissue structures. The rapid development of deep learning provides a good solution to generate virtual staining images to significantly reduce the time and labor costs associated with histochemical staining. However, most existing methods need to train a special model for every two stains, which consumes a lot of computing resources with the increasing of staining types. To address this problem, we propose an unsupervised multi-domain stain transfer method, GramGAN, which realizes the progressive transfer through cascaded Style-Guided blocks. For each Style-Guided block, we design a style encoding dictionary to characterize and store all the staining style information. In addition, we propose a Rényi entropy-based regularization term to improve the discrimination ability of different styles. The experimental results show that our method can realize accurate transferring among multiple staining styles with better performance. Furthermore, we build and publish a special stained image dataset suitable for glomeruli segmentation (including H&E staining), where the accuracy of glomeruli detection and segmentation can be significantly improved after transferring H&E-stained images to PAS-stained and PASM-stained ones by our method. The code is publicly available at: https://github.com/xianchaoguan/GramGAN .

Tissue segmentation is a critical task in computational pathology due to its desirable ability to indicate the prognosis of cancer patients. Currently, numerous studies attempt to use image-level labels to achieve pixel-level segmentation to reduce the need for fine annotations. However, most of these methods are based on class activation map, which suffers from inaccurate segmentation boundaries. To address this problem, we propose a novel weakly-supervised tissue segmentation framework named PistoSeg, which is implemented under a fully-supervised manner by transferring tissue category labels to pixel-level masks. Firstly, a dataset synthesis method is proposed based on Mosaic transformation to generate synthesized images with pixel-level masks. Next, considering the difference between synthesized and real images, this paper devises an attention-based feature consistency, which directs the training process of a proposed pseudo-mask refining module. Finally, the refined pseudo-masks are used to train a precise segmentation model for testing. Experiments based on WSSS4LUAD and BCSS-WSSS validate that PistoSeg outperforms the state-of-the-art methods. The code is released at https://github.com/Vison307/PistoSeg.

Inertial sensors, as motion-sensing components, are widely used in inertial navigation, aerospace, and consumer electronics. Their wide applications and severe errors form a sharp contradiction, which attracts considerable attention. Wavelet is hailed as the mathematical signal microscope due to the diverse wavelet basis functions. However, complicated noise types and application scenarios of inertial sensors make selecting wavelet basis perplexing. To this end, we propose a wavelet encoding network (WENet), which intelligently selects the appropriate wavelet for variable inertial signals by representing wavelet characteristics through the devised category representation mechanism (CRM). Furthermore, CRM introduces a feature supervision effect, which imposes interpretability on a black-box network and forces it to provide direct supervision for other network structures. This supervision strategy is closer and more effective than the supervision provided by the output end, which is far and needs to go through backpropagation. The proposed WENet has the reliability of the model-driven method and the flexibility of the data-driven method. As a weakly supervised method, the WENet achieves the best performance among all signal improvement methods, including fully supervised ones. After being enhanced by WENet, the low-cost sensor signal can perform accurate spatial trajectory reconstruction, which was once considered an impossible task.

Automatic polyp segmentation in endoscopic images is critical for the early diagnosis of colorectal cancer. Despite the availability of powerful segmentation models, two challenges still impede the accuracy of polyp segmentation algorithms. Firstly, during a colonoscopy, physicians frequently adjust the orientation of the colonoscope tip to capture underlying lesions, resulting in viewpoint changes in the colonoscopy images. These variations increase the diversity of polyp visual appearance, posing a challenge for learning robust polyp features. Secondly, polyps often exhibit properties similar to the surrounding tissues, leading to indistinct polyp boundaries. To address these problems, we propose a viewpoint-aware framework named VANet for precise polyp segmentation. In VANet, polyps are emphasized as a discriminative feature and thus can be localized by class activation maps in a viewpoint classification process. With these polyp locations, we design a viewpoint-aware Transformer (VAFormer) to alleviate the erosion of attention by the surrounding tissues, thereby inducing better polyp representations. Additionally, to enhance the polyp boundary perception of the network, we develop a boundary-aware Transformer (BAFormer) to encourage self-attention towards uncertain regions. As a consequence, the combination of the two modules is capable of calibrating predictions and significantly improving polyp segmentation performance. Extensive experiments on seven public datasets across six metrics demonstrate the state-of-the-art results of our method, and VANet can handle colonoscopy images in real-world scenarios effectively. The source code is available at https://github.com/1024803482/Viewpoint-Aware-Network.

Automated classification of lymph node metastasis (LNM) plays an important role in the diagnosis and prognosis. However, it is very challenging to achieve satisfactory performance in LNM classification, because both the morphology and spatial distribution of tumor regions should be taken into account. To address this problem, this paper proposes a two-stage dMIL-Transformer framework, which integrates both the morphological and spatial information of the tumor regions based on the theory of multiple instance learning (MIL). In the first stage, a double Max-Min MIL (dMIL) strategy is devised to select the suspected top-K positive instances from each input histopathology image, which contains tens of thousands of patches (primarily negative). The dMIL strategy enables a better decision boundary for selecting the critical instances compared with other methods. In the second stage, a Transformer-based MIL aggregator is designed to integrate all the morphological and spatial information of the selected instances from the first stage. The self-attention mechanism is further employed to characterize the correlation between different instances and learn the bag-level representation for predicting the LNM category. The proposed dMIL-Transformer can effectively deal with the thorny classification in LNM with great visualization and interpretability. We conduct various experiments over three LNM datasets, and achieve 1.79%-7.50% performance improvement compared with other state-of-the-art methods.