Transformers have been used for 3D human pose estimation with excellent performance; however, most transformers focus on encoding the global spatio-temporal correlation of all joints in the human body and there are few studies on the local Spatio-temporal correlation of each joint in the human body. In this paper, we propose a Global and Local Spatio-Temporal Encoder (GLSTE) to model the Spatio-temporal correlation. Specifically, a Global Spatial Encoder (GSE) and a Global Temporal Encoder (GTE) are constructed to capture the global spatial information of all joints in a single frame and the global temporal information of all frames, respectively. A Local Spatio-Temporal Encoder (LSTE) is constructed to capture the spatial and temporal information of each joint in the local N frames. Furthermore, we propose a parallel attention module with weight sharing to better incorporate spatial and temporal information into each node simultaneously. Extensive experiments show that GLSTE outperforms state-of-the-art methods with fewer parameters and less computational overhead on two challenging datasets: Human3.6M and MPI-INF-3DHP. Especially in the evaluation of Human3.6M dataset, the results of our method with 27 frames as input are better than the vast majority of recent SOTA methods with 81 and 243 frames as input, which indicates that the model can learn more useful information with smaller inputs.
Traditional real-time object detection networks deployed in unmanned aerial vehicles (UAVs) struggle to extract features from small objects in complex backgrounds with occlusions and overlapping objects. To address this challenge, we propose FM-RTDETR, a real-time object detection algorithm optimized for small object detection. We redesign the encoder of RT-DETRv2 by integrating the Feature Aggregation and Diffusion Network (FADN), improving the algorithm's ability to capture contextual information. Subsequently, we introduce the Parallel Atrous Mamba Feature Fusion Module (PAMFFM), which combines shallow and deep semantic information to better capture small object features. Furthermore, we propose the Cross-stage Enhanced Feature Fusion Module (CEFFM), merging features for small objects to provide richer and more detailed information. Finally, we propose STIoU Loss, which incorporates a penalty term to adjust the scaling of the loss function, thereby improving detection granularity for small objects. FM-RTDETR achieves AP$_{50}$ scores of 54.0% and 56.3% on the VisDrone2019-DET and AI-TOD datasets. Compared with other state-of-the-art methods, our method shows great potential in small object detection from drones. The code is available at https://github.com/Yyc1999super/FM-RTDETR.
In conditions of foggy weather, challenges such as low light, blurred imagery, and dense fog that obscures target objects are prevalent. Moreover, computing resources are limited on edge devices. To tackle these challenges, a novel real-time detection algorithm GMS-YOLO for pedestrians and vehicles is proposed based on YOLOv10, which overcomes the semantic bottleneck of the model and enhances its detection performance. A novel Ghost Multi-Scale Convolution (GMSConv) module is constructed, serving as the Ghost Multi-Scale feature extraction backbone network (GMS-Net). The Shape Consistent Intersection over Union (SCIoU) is introduced as the localization loss function, which takes into account the influence of the attributes of the regression box in the loss computation. Additionally, a Compensatory Consistency Matching Metric (CCMM) formula is designed to reduce the sensitivity of the original metric to IoU and regression scores. The GMS-YOLO algorithm has a lightweight structure, achieving FPS of 94 and 92 during the detection phase at the ’n’ and ’s’ sizes, respectively. Furthermore, we have deployed the model on Jetson Nano hardware, and the inference speed is also quite encouraging. We validated the effectiveness of the algorithm on the Foggy Cityscapes, RTTS, VOC2007-fog, and VOC2012-fog datasets. Experimental results indicate that GMS-YOLO outperforms the baseline model, with a mean Average Precision (mAP) improvement of 6.3% and 5.5% for the ’n’ and ’s’ scales, respectively. Consequently, the proposed GMS-YOLO algorithm not only demonstrates superior detection performance but also maintains a relatively low model complexity, significantly enhancing the efficiency and accuracy of object detection tasks in foggy environments. The source code for our algorithm is available at: https://github.com/Fwdchina/GMS-YOLO.