Remote sensing semantic segmentation plays a significant role in various applications such as environmental monitoring, land use planning, and disaster response. CNNs have been dominating remote sensing semantic segmentation. However, due to the limitations of convolution operations, CNNs cannot effectively model global context. The success of Transformers in the NLP domain provides a new solution for global context modeling. Inspired by Swin Transformer, we propose a novel remote sensing semantic segmentation model called CSTUNet. This model employs a dual-encoder structure consisting of a CNN-based main encoder and a Swin Transformer-based auxiliary encoder. We first utilize a detail-structure preservation module (DPM) to mitigate the loss of detail and structure information caused by Swin Transformer downsampling. Then we introduce a spatial feature enhancement module (SFE) to collect contextual information from different spatial dimensions. Finally, we construct a position-aware attention fusion module (PAFM) to fuse contextual and local information. Our proposed model obtained 70.75% MIoU on the ISPRS-Vaihingen dataset and 77.27% MIoU on the ISPRS-Potsdam dataset.

Most of the existing semantic segmentation approaches with image-level class labels as supervision, highly rely on the initial class activation map (CAM) generated from the standard classification network. In this paper, a novel ‘`Progressive Patch Learning’' approach is proposed to improve the local details extraction of the classification, producing the CAM better covering the whole object rather than only the most discriminative regions as in CAMs obtained in conventional classification models. ‘`Patch Learning’' destructs the feature maps into patches and independently processes each local patch in parallel before the final aggregation. Such a mechanism enforces the network to find weak information from the scattered discriminative local parts, achieving enhanced local details sensitivity. ‘`Progressive Patch Learning’' further extends the feature destruction and patch learning to multi-level granularities in a progressive manner. Cooperating with a multi-stage optimization strategy, such a ‘`Progressive Patch Learning’' mechanism implicitly provides the model with the feature extraction ability across different locality-granularities. As an alternative to the implicit multi-granularity progressive fusion approach, we additionally propose an explicit method to simultaneously fuse features from different granularites in a single model, further enhancing the CAM quality on the full object coverage. Our proposed method achieves outstanding performance on the PASCAL VOC 2012 dataset e.g., with 69.6% mIoU on the test set), which surpasses most existing weakly supervised semantic segmentation methods.

Utilizing data acquired by multiple wearable sensors can usually guarantee more accurate recognition for deep learning based human activity recognition. However, an increased number of sensors bring high processing cost, influencing real-time activity monitoring. Besides, existing methods rarely consider the interpretability of the recognition model in aspects of both the importance of the sensors and features, causing a gap between deep learning and their extendability in real-world scenario. In this paper, we cast the classical fused lasso model into a deep neural network, proposing a deep fused Lasso net (dfLasso-Net), which can perform sensor selection, feature selection and HAR in one end-to-end structure. Specifically, a two-level weight computing module (TLWCM) consisting of a senor weight net and a feature weight net is designed to measure the importance of sensors and features. In sensor weight net, spatial smoothness between physical channels within each sensor is considered to maximize the usage of selected sensors. And the feature weight net is able to maintain the physical meaning of the hand-crafted features through feature selection inside the sensors. By combining with the learning module for classification, HAR can be performed. We test dfLasso-Net on three multi-sensor based HAR datasets, demonstrating that dfLasso-Net achieves better recognition accuracy with the least number of sensors and provides good model interpretability by visualizing the weights of the sensors and features. Last but not least, dflasso-Net can be used as an effective filter-based feature selection approach with much flexibility.

With the popularity and development of Internet of Things (IoT) technology, human activity recognition using IoT devices such as wearable sensors can be implemented for various applications. Due to the complexity of activity recognition, multiple homogeneous or heterogeneous sensors are used to obtain excessive information in most wearable activity recognition systems. However, the increased number of sensors and the way of multichannel signal data bring huge challenges to human activity recognition tasks. How to select suitable sensor channels to balance the computational complexity and recognition accuracy has become a major issue. In this article, we extend the sparse group Lasso mechanism to human activity recognition tasks and propose a hybrid attention-based multi-sensor pruning and feature selection deep neural network, called HAP-DNN. This architecture is able to further perform feature selection on the basis of sensor pruning. HAP-DNN consists of three detachable modules: 1) a feature compression & reconstruction module for sensor feature information fusion and restoration; 2) a feature weight calculation module for calculating sensor channel weights and feature weights; and 3) a learning module for classification, which can be regarded as a filter feature selection method. Four public activity recognition data sets are used to verify our proposed architecture, and the experimental results show that HAP-DNN achieves the best classification performance with the least number of retained feature channels.