Road extraction from high-resolution remote sensing images has long been a focal and challenging research topic in the field of computer vision. Accurate extraction of road networks holds extensive practical value in various fields, such as urban planning, traffic monitoring, disaster response and environmental monitoring. With rapid development in the field of computational intelligence, particularly breakthroughs in deep learning technology, road extraction technology has made significant progress and innovation. This paper provides a systematic review of deep learning-based methods for road extraction from remote sensing images, focusing on analyzing the application of computational intelligence technologies in improving the precision and efficiency of road extraction. According to the type of annotated data, deep learning-based methods are categorized into fully supervised learning, semi-supervised learning, and unsupervised learning approaches, each further divided into more specific subcategories. They are comparatively analyzed based on their principles, advantages, and limitations. Additionally, this review summarizes the metrics used to evaluate the performance of road extraction models and the high-resolution remote sensing image datasets applied for road extraction. Finally, we discuss the main challenges and prospects for leveraging computational intelligence techniques to enhance the precision, automation, and intelligence of road network extraction.

Vision Transformer, which performs well in various vision tasks, encounters a bottleneck in skeleton-based action recognition and falls short of advanced GCN-based methods. The root cause is that the current skeleton transformer depends on the self-attention mechanism of the complete channel of the global joint, ignoring the highly discriminative differential correlation within the channel, so it is challenging to learn the expression of the multivariate topology dynamically. To tackle this, we present Skeleton MixFormer, an innovative spatio-temporal architecture to effectively represent the physical correlations and temporal interactivity of the compact skeleton data. Two essential components make up the proposed framework: 1) Spatial MixFormer. The channel-grouping and mix-attention are utilized to calculate the dynamic multivariate topological relationships. Compared with the full-channel self-attention method, Spatial MixFormer better highlights the channel groups' discriminative differences and the joint adjacency's interpretable learning. 2) Temporal MixFormer, which consists of Multiscale Convolution, Temporal Transformer and Sequential Holding Module. The multivariate temporal models ensure the richness of global difference expression and realize the discrimination of crucial intervals in the sequence, thereby enabling more effective learning of long and short-term dependencies in actions. Our Skeleton MixFormer demonstrates state-of-the-art (SOTA) performance across seven different settings on four standard datasets, namely NTU-60, NTU-120, NW-UCLA, and UAV-Human. Related code will be available on https://github.com/ElricXin/Skeleton-MixFormer.

The accurate detection and extraction of roads using remote sensing technology are crucial to the development of the transportation industry and intelligent perception tasks. Recently, in view of the advantages of CNNs in feature extraction, its related road extraction methods have been proposed successively. However, due to the limitation of kernel size, they perform less effectively at capturing long-range information and global context, which are crucial for road targets distributed over long distances and highly structured. To deal with this problem, a novel model named RoadFormer with a Swin Transformer as the backbone is developed in this paper. Firstly, to extract long-range information effectively, a Swin Transformer multi-scale encoder is adopted in our model. Secondly, to enhance the feature representation capability of the model, we design an innovative bottleneck module, in which the spatial and channel separable convolution is employed to obtain fine-grained and globe features, and then a dilated block is connected after the spatial convolution module to capture more integrated road structures. Finally, a lightweight decoder consisting of transposed convolution and skip connection generates the final extraction results. Extensive experimental results confirm the advantages of RoadFormer on the Deepglobe and Massachusetts datasets. The comparative results of visualization and quantification demonstrate that our model outperforms comparable methods.

Different from large-scale classification tasks, fine-grained visual classification is a challenging task due to two critical problems: 1) evident intra-class variances and subtle inter-class differences, and 2) overfitting owing to fewer training samples in datasets. Most existing methods extract key features to reduce intra-class variances, but pay no attention to subtle inter-class differences in fine-grained visual classification. To address this issue, we propose a loss function named exploration of class center, which consists of a multiple class-center constraint and a class-center label generation. This loss function fully utilizes the information of the class center from the perspective of features and labels. From the feature perspective, the multiple class-center constraint pulls samples closer to the target class center, and pushes samples away from the most similar nontarget class center. Thus, the constraint reduces intra-class variances and enlarges inter-class differences. From the label perspective, the class-center label generation utilizes class-center distributions to generate soft labels to alleviate overfitting. Our method can be easily integrated with existing fine-grained visual classification approaches as a loss function, to further boost excellent performance with only slight training costs. Extensive experiments are conducted to demonstrate consistent improvements achieved by our method on four widely-used fine-grained visual classification datasets. In particular, our method achieves state-of-the-art performance on the FGVC-Aircraft and CUB-200-2011 datasets.

Human action recognition (HAR), which aims to recognize and understand individual actions and intentions, has rapidly become a research hotspot in computer vision. Compared with other data modalities, skeleton data offers more efficient node semantics and more coherent spatio-temporal motion patterns, effectively reducing the impact of lighting and background changes. In recent years, many researchers have focused on skeleton-based action recognition methods and have made significant progress. However, we believe that the current skeleton-based action recognition methods still face three major challenges: 1) reducing reliance on expensive labeled data while maintaining model performance; 2) enabling the model to understand and recognize new behavior classes with a limited number of samples; and 3) addressing the challenges posed by the lack of skeleton information in single-modality spatio-temporal motion representation learning. Based on these challenges, we conduct a comprehensive review of the existing skeleton-based action recognition methods. Additionally, we provide an extensive review and analysis of publicly available action recognition datasets. This review aims to offer researchers a comprehensive perspective, stimulate more innovative ideas, and promote the application and breakthrough of skeleton action recognition in a wider range of computer vision tasks.

Skeleton-based action recognition is a crucial but challenging task in the application of engineering algorithms. However, due to the inaccurate estimation quality, certain joints that should theoretically lack dynamic information show irregular jitter, which affects the recognition accuracy. In this paper, we propose a Cascade Interaction Spatial-temporal transformer Network ( CI-STFormer ) to address abnormal joint jitter and multi-scale aggregation of features effectively. The network comprises three parts: (1) Spatial sparse-level optimization and interaction module. Relying on the topology map, the sparse-level interaction adjacency matrix is constructed through cascading interaction to mask the jitter non-discriminative joints, and then the transformer is applied to realize the sparse-level internal feature update. (2) Spatial primal-level fusion and interaction module. Multi-head self-attention is employed to enhance the original feature representation. Then, the sparse-level update features are cascaded to construct the scale-cascaded interaction adjacency matrix to achieve the balance of noise caused by the jitter and the fusion of different scale features. (3) Temporal domain scale-level TCN combines the multi-scale filter and the temporal self-attention channel interaction algorithm to extract different temporal features and the interaction and update of global temporal information. The experimental results show that the proposed method performs excellently on the four datasets of NTU RGB+D 60, NTU RGB+D 120, UAV-Human, and NW-UCLA and achieves state-of-the-art performance on the transformer-based track. Related code will be available on https://github.com/Xdu-Liu/CI-STFormer.git.