Multispectral object detection for autonomous driving is multi-object localization and classification task on visible and thermal modalities. In this scenario, modality differences lead to the lack of object information in a single modality and the misalignment of cross-modality information. To alleviate these problems, most existing methods extract information based on a single scale ( e.g ., these methods mainly focus on detecting significant cars or pedestrians), which leads to insufficient performance in capturing multi-scale discriminative information ( e.g ., small bicycles and blurred pedestrians) and safety hazards in the driving process. In this paper, we propose a Multi-Scale Aggregation Network (MSANet) consisting of two parts Multi-Scale Aggregation Transformer (MSAT) and the Cross-modal Merging Fusion Mechanism (CMFM), which combined with the advantages of Transformer and CNN to extract rich image information from two modalities by mining both local and global context dependencies. Firstly, to reduce the lack of information in a single modality, we design a novel MSAT module to extract rich details and texture from multi-scale. Secondly, to alleviate feature misalignment caused by modality differences, the CMFM is utilized to aggregate complementary information on multiple levels. Comprehensive experiments on two benchmarks demonstrate that our approach shows better results than several state-of-the-art methods. The code is available at https://github.com/ysh-strive/MSANet .

Due to changes in illumination, adverse weather conditions, and interference from signs similar to real traffic signs, the false detection of traffic signs is possible. Nevertheless, in order to improve the detection effect of small targets, baseline SSD (single shot multibox detector) adopts a multi-scale feature detection method to improve the detection effect to some extent. The detection effect of small targets is improved, but the number of calculations needed for the baseline SSD network is large. To this end, we propose a lightweight SSD network algorithm. This method uses some 1 × 1 convolution kernels to replace some of the 3 × 3 convolution kernels in the baseline network and deletes some convolutional layers to reduce the calculation load of the baseline SSD network. Then the color detection algorithm based on the phase difference method and the connected component calculation are used to further filter the detection results, and finally, the data enhancement strategy based on the image appearance transformation is used to improve the balance of the dataset. The experimental results show that the proposed method is 3% more accurate than the baseline SSD network, and more importantly, the detection speed is also increased by 1.2 times.

Text-based Person Retrieval (TPR) plays a pivotal role in video surveillance systems for safeguarding public safety. As a fine-grained retrieval task, TPR faces the significant challenge of precisely capturing highly discriminative features across image and text modalities. Existing methods primarily focus on establishing modality-shared feature spaces to bridge cross-modal discrepancies. However, these methods are prone to disturbances from irrelevant information, such as background noises in the visual modality, and often over-emphasize specific local regions while neglecting the capture of diverse discriminative modal features, thereby limiting the robustness of cross-modal matching. In this paper, we introduce a novel framework, termed the Diverse Co-saliency Feature Learning Network (DCFL), which mines the co-saliency information between image and text modalities and enhances the diversity of cross-modal discriminative features while mitigating the interference of noise. Specifically, to construct cross-modal co-saliency features, we devise the Intra-modal Saliency Feature Learning (ISFL) and Cross-modal Saliency Feature Matching (CSFM) modules. ISFL employs a weighted mask mechanism to guide the model in reducing the impact of noise information in both modalities. Complementing ISFL, CSFM establishes consistent relationships between saliency features across modalities, leveraging text descriptions to align pedestrian-relevant visual regions. Furthermore, we propose the Diverse Co-saliency Feature Mining (DCFM) to bolster the diversity of discriminative co-saliency features across both image and text modalities. This module integrates a diversity regularization term, enabling the extraction of varied visual cues and capturing comprehensive features of the target individual. Extensive benchmark experiments demonstrate a substantial superiority of our approach over the state-of-the-art methods. Our code is available at https://github.com/ysh-strive/DCFL

Occlusion is a key challenge in real-world person reidentification (ReID), as partial occlusions often remove critical identity cues. Most existing methods focus on feature extraction for visible regions while ignoring the holistic contextual information of pedestrians, leading to incomplete feature representations. This issue is particularly severe in visible-infrared ReID (VI-ReID), where spatial inconsistencies of occluded regions and modality-specific feature degradation jointly exacerbate feature misalignment, but has received little attention in previous studies. Therefore, we propose an Occlusion-Aware Cross-Modality Completion Network (OCCNet) to explicitly complete missing identity cues, which integrates three complementary components. The Graph-guided Feature Completion Transformer (GFCT) embeds a dynamically constructed semantic graph into self-attention to complete occluded tokens within each modality, while the Semantic-Aware Region Completion Module (SRCM) aggregates tokens into coherent regions and adaptively transfers complementary cues across modalities. To ensure semantic coherence, a Multi-Distribution Consistency Alignment (MCA) loss is employed to regularize token- and region-level completions toward a unified distribution. By integrating these components, OCCNet recovers missing identity information at multiple levels and mitigates modality gaps caused by occlusion. To evaluate its effectiveness under realistic occlusion scenarios, we first construct two realistic occluded benchmarks, Occluded-SYSU-MM01 and Occluded-RegDB, on which extensive experiments demonstrate that our OCCNet outperforms state-of-the-art methods.

Accurately segmenting the optic disk (OD) and optic cup (OC) on retinal fundus images is important for treating glaucoma. With the development of deep learning, some CNN-based methods have been implemented to segment OD and OC, but it is difficult to accurately segment OD and OC boundaries affected by blood vessels and the lesion area. To this end, we propose a novel boundary-enhanced adaptive context network (BEAC-Net) for OD and OC segmentation. Firstly, a newly designed efficient boundary pixel attention (EBPA) module enhances pixel-by-pixel feature capture to collect the boundary contextual information of OD and OC in the horizontal and vertical directions. In addition, background noise makes segmenting boundary pixels difficult. To this end, an adaptive context module (ACM) was designed, which simultaneously learns local-range and long-range information to capture richer context. Finally, BEAC-Net adaptively integrates the feature maps from different levels using the attentional feature fusion (AFF) module. In addition, we provide a high-quality retinal fundus image dataset named the 66 Vision-Tech dataset, which advances the field of diagnostic glaucoma. Our proposed BEAC-Net was used to perform extensive experiments on the RIM-ONE-v3, DRISHTI-GS, and 66 Vision-Tech datasets. In particular, BEAC-Net achieved a Dice coefficient of 0.8267 and an IoU of 0.8138 for OD segmentation and a Dice coefficient of 0.8057 and an IoU value of 0.7858 for OC segmentation on the 66 Vision-Tech dataset, achieving state-of-the-art segmentation results.

The thermal imaging pedestrian-detection system has excellent performance in different lighting scenarios, but there are problems regarding weak texture, object occlusion, and small objects. Meanwhile, large high-performance models have higher latency on edge devices with limited computing power. To solve the above problems, in this paper, we propose a real-time thermal imaging pedestrian-detection method for edge computing devices. Firstly, we utilize multi-scale mosaic data augmentation to enhance the diversity and texture of objects, which alleviates the impact of complex environments. Then, the parameter-free attention mechanism is introduced into the network to enhance features, which barely increases the computing cost of the network. Finally, we accelerate multi-channel video detection through quantization and multi-threading techniques on edge computing devices. Additionally, we create a high-quality thermal infrared dataset to facilitate the research. The comparative experiments on the self-built dataset, YDTIP, and three public datasets, with other methods show that our method also has certain advantages.