Given two multitemporal aerial images, semantic change detection (SCD) aims to locate the land-cover variations and identify their change types with pixelwise boundaries. This problem is vital in many earth vision-related tasks, such as precise urban planning and natural resource management. Existing state-of-the-art algorithms mainly identify the changed pixels by applying homogeneous operations on each input image and comparing the extracted features. However, in changed regions, totally different land-cover distributions often require heterogeneous feature extraction procedures for images acquired at different times. In this article, we present an asymmetric Siamese network (ASN) to locate and identify semantic changes through feature pairs obtained from modules of widely different structures, which involves areas of various sizes and applies different quantities of parameters to factor in the discrepancy across land-cover distributions during different times. To better train and evaluate our model, we create a large-scale well-annotated SEmantic Change detectiON Dataset (SECOND), while an adaptive threshold learning (ATL) module and a separated kappa (SeK) coefficient are proposed to alleviate the influences of label imbalance in model training and evaluation. The experimental results demonstrate that the proposed model can stably outperform the state-of-the-art algorithms with different encoder backbones.

Fatigue driving is a common issue that often leads to traffic accidents, which has motivated numerous automatic driving fatigue detection methods based on various sources, especially reliable physiological signals. However, it still faces the challenges of accuracy, robustness and practicality, especially for the cross-subject detection. The fusion of multi-modality data can improve the effective estimation of driving fatigue. In this work, we take the advantages of user-friendly and multi-modality signals to build a Multi-Modality Attention Network (MMA-Net) for driver fatigue detection with frontal electroencephalography (EEG), electrodermal activity (EDA) and photoplethysmography (PPG) signals for a hybrid. Specifically, a signal adaptive coding module (SAC-M) has been constructed to fully excavate spatial-temporal information of signals, combining with an attention-based feature dissimilation module (AFD-M) to further obtain key comprehensive features. In addition, the performances of baseline models and state-of-the-art methods on signal sources with different window lengths are also compared. The cross-subject experiment is performed on two groups of 14 participants in the driving simulation experiment. The experimental results prove the superiority of our proposed method. It is possible to use the MMA-Net for driver fatigue detection with user-friendly multi-modality signals such as our selected frontal EEG, EDA and PPG in real-world applications.

Targeting at depicting land covers with pixelwise semantic categories, semantic segmentation in remote sensing images needs to portray diverse distributions over vast geographical locations, which is difficult to be achieved by the homogeneous pixelwise forward paths in the architectures of existing deep models. Although specific algorithms have been designed to select pixelwise adaptive forward paths for natural image analysis, it still lacks theoretical supports on how to obtain optimal selections. In this article, we provide mathematical analyses in terms of the parameter optimization, which guides us to design a method called hidden path selection network (HPS-Net). With the help of hidden variables deriving from an extra mini-branch, HPS-Net is able to tackle the inherent problem about inaccessible global optimums by adjusting the direct relationships between feature maps and pixelwise path selections in existing algorithms, which we call hidden path selection. For the better training and evaluation, we further refine and expand the 5-class Gaofen image dataset (GID-5) to a new one with 15 land-cover categories, i.e., GID-15. The experimental results on both GID-5 and GID-15 demonstrate that the proposed modules can stably improve the performance of different deep structures, which validates the proposed mathematical analyses.

Traditional dehazing approaches that rely on prior knowledge exhibit limited efficacy when confronted with the intricacies of real-world hazy environments. While learning-based dehazing techniques necessitate large-scale datasets for effective model training, the acquisition of these datasets is time-consuming and laborious, and the resulting models may encounter a domain shift when processing real-world hazy images. To overcome the limitations of prior-based and learning-based dehazing methods, we propose a self-supervised remote sensing (RS) image-dehazing network based on zero-shot learning, where the self-supervised process avoids dense dataset requirements and the learning-based structures refine the artifacts in extracted image priors caused by complex real-world environments. The proposed method has three stages. The first stage involves pre-processing the input hazy image by utilizing a prior-based dehazing module; in this study, we employed the widely recognized dark channel prior (DCP) to obtain atmospheric light, a transmission map, and the preliminary dehazed image. In the second stage, we devised two convolutional neural networks, known as RefineNets, dedicated to enhancing the transmission map and the initial dehazed image. In the final stage, we generated a hazy image using the atmospheric light, the refined transmission map, and the refined dehazed image by following the haze imaging model. The meticulously crafted loss function encourages cycle-consistency between the regenerated hazy image and the input hazy image, thereby facilitating a self-supervised dehazing model. During the inference phase, the model undergoes training in a zero-shot manner to yield the haze-free image. These thorough experiments validate the substantial improvement of our method over the prior-based dehazing module and the zero-shot training efficiency. Furthermore, assessments conducted on both uniform and non-uniform RS hazy images demonstrate the superiority of our proposed dehazing technique.

Image dehazing is crucial for improving the advanced applications on remote sensing (RS) images. However, collecting paired RS images to train the deep neural networks (DNNs) is scarcely available, and the synthetic datasets may suffer from domain-shift issues. In this paper, we propose a zero-shot RS image dehazing method based on a re-degradation haze imaging model, which directly restores the haze-free image from a single hazy image. Based on layer disentanglement, we design a dehazing framework consisting of three joint sub-modules to disentangle the hazy input image into three components: the atmospheric light, the transmission map, and the recovered haze-free image. We then generate a re-degraded hazy image by mixing up the hazy input image and the recovered haze-free image. By the proposed re-degradation haze imaging model, we theoretically demonstrate that the hazy input and the re-degraded hazy image follow a similar haze imaging model. This finding helps us to train the dehazing network in a zero-shot manner. The dehazing network is optimized to generate outputs that satisfy the relationship between the hazy input image and the re-degraded hazy image in the re-degradation haze imaging model. Therefore, given a hazy RS image, the dehazing network directly infers the haze-free image by minimizing a specific loss function. Using uniform hazy datasets, non-uniform hazy datasets, and real-world hazy images, we conducted comprehensive experiments to show that our method outperforms many state-of-the-art (SOTA) methods in processing uniform or slight/moderate non-uniform RS hazy images. In addition, evaluation on a high-level vision task (RS image road extraction) further demonstrates the effectiveness and promising performance of the proposed zero-shot dehazing method.

The calculation of Tumor Stroma Ratio (TSR) is a challenging medical issue that could improve predictions of neoadjuvant chemotherapy benefits and patient prognoses. Although several studies on breast cancer and deep learning methods have achieved promising results, the drawbacks that pixel-level semantic segmentation processes could not extract core tumor regions containing both tumor pixels and stroma pixels make it difficult to accurately calculate TSR. In this paper, we propose a Vague-Segment Technique (VST) consisting of a designed SwinV2UNet module and a modified Suzuki algorithm. Specifically, the SwinV2UNet identifies tumor pixels and generate pixel-level classification results, based on which the modified Suzuki algorithm extracts the contour of core tumor regions in terms of cosine angle. Through this way, VST obtains vaguely segmentation results of core tumor regions containing both tumor pixels and stroma pixels, where the TSR could be calculated by the formula of Intersection over Union (IOU). For the training and evaluation, we utilize the well-known The Cancer Genome Atlas (TCGA) database to create an annotated dataset, while 150 images with TSR annotations from real cases are also collected. The experimental results illustrate that the proposed VST could generate better tumor identification results compared with state-of-the-art methods, where the extracted core tumor regions lead to more consistencies of calculated TSR with senior experts compared to junior pathologists. The experimental results demonstrate the superiority of our proposed pipeline, which has promise for future clinical application.