Aiming at the problems of serious information redundancy, complex inter-modal information interaction, and difficult multimodal fusion faced by the audio–visual speech recognition system when dealing with complex multimodal information, this paper proposes an adaptive fusion transformer algorithm (AFT-SAM) based on a sparse attention mechanism. The algorithm adopts the sparse attention mechanism in the feature-encoding process to reduce excessive attention to non-important regions and dynamically adjusts the attention weights through adaptive fusion to capture and integrate the multimodal information more effectively and reduce the impact of redundant information on the model performance. Experiments are conducted on the audio–visual speech recognition dataset LRS2 and compared with other algorithms, and the experimental results show that the proposed algorithm in this paper has significantly lower WERs in the audio-only, visual-only, and audio–visual bimodal cases.
Convolutional neural networks (CNN) and Vision Transformers (ViT) have long been the main backbone networks for visual classification in the field of deep learning. Although ViT has recently received more attention than CNN due to its excellent fitting ability, their scalability is largely limited by the quadratic complexity of attention computation. For the determination of diabetic retinopathy, the fundus lesions as well as the width, angle, and branching pattern of retinal blood vessels are characterized, inspired by the ability of Mamba and VMamba to efficiently model long sequences, VMamba-m is proposed in this paper. This is a generalized visual skeleton model designed to reduce computational complexity to linear while retaining the advantageous features of ViTs. By modifying the cross-entropy loss function, we enhance the model’s attention to rare categories, especially in large-scale multi-category classification tasks. In order to enhance the adaptability of the VMamba-m model in processing visual data, we introduce the se channel attention mechanism, which enables the model to learn features in the channel dimension and form the importance of each channel. Finally, different weights are assigned to each channel through the incentive part. In addition to this, this paper further improves the implementation details and architectural design by introducing a novel attention mechanism implemented based on the local windowing method, which aims to optimize the model’s ability in processing long sequence data to enhance the performance of VMamba-m and improve its inference speed. Extensive experimental results show that VMamba-m performs well in the retinopathy V classification task, and it has significant advantages in terms of accuracy and computation time over existing benchmark models.
Aiming at the problems of large search space, unstable computational efficiency, and lack of safety of generated paths in complex environments of traditional HybridA* algorithms, this paper proposes an improved HybridA* algorithm based on Voronoi diagrams and safe corridors (GCHybridA*) to overcome these challenges. The method first reduces ineffective node expansion by constructing a Voronoi path away from obstacles and smoothing it, followed by selecting key guidance points to provide stage-like goals for path search. Then, an innovative safe corridor is generated and the path search is restricted to the safe corridor area to guarantee the safety of the path, and an adaptive step-size mechanism is designed to balance the search efficiency and path quality. The experimental results show that the GCHybridA* algorithm significantly outperforms the conventional HybridA* algorithm, with an average reduction of 83.7% in node expansions while maintaining zero potential collision points across all four typical maps. This study provides an innovative and robust solution for efficient and safe path planning in autonomous driving systems. This study provides an innovative and robust solution for global path planning in autonomous driving systems, focusing on static environment navigation with safety guarantees.
As autonomous driving technology progresses, efficient and accurate object detectors are able to detect pedestrians, vehicles, road signs, and obstacles in real time, thereby enhancing driving safety and serving as a part of autonomous driving. However, the performance of such object detectors is limited and cannot be leveraged to satisfy modern autonomous driving systems. To address this issue, we develop an object detection network for autonomous driving scenarios, SST-YOLO, which is based on YOLOv8. First, we propose a Sobel Convolution & Convolution (SCC) module to enhance the backbone, which incorporates a SobelConv branch to explicitly model gradient-based edge information and improve structural feature representation. In addition, we replace the original path aggregation feature pyramid network (PAFPN) with a Small Object Augmentation Pyramid Network (SOAPN), which integrates SPDConv and CSP-OmniKernel modules to strengthen multi-scale feature fusion and enhance small object representation. Finally, a Task-Adaptive Decomposition & Alignment Head (TADAHead) is designed, which employs task decomposition, dynamic deformable convolution, and classification-aware modulation to decouple tasks and achieve adaptive spatial alignment, thereby improving detection accuracy and robustness in complex scenarios. Experiments on the public autonomous driving dataset KITTI show that our proposed method outperforms the baseline YOLOv8 model. Compared with the baseline results, mAP@0.5:0.95 ranges from 65.1% to 69.2%, which indicates that the proposed SST-YOLO network can achieve object detection for autonomous cars.