In recent years, the rise of low-cost mini rotary-wing drone technology across diverse sectors has emphasized the crucial role of object detection within drone aerial imagery. Low-cost mini rotary-wing drones come with intrinsic limitations, especially in computational power. Drones come with intrinsic limitations, especially in resource availability. This context underscores an urgent need for solutions that synergize low latency, high precision, and computational efficiency. Previous methodologies have primarily depended on high-resolution images, leading to considerable computational burdens. To enhance the efficiency and accuracy of object detection in drone aerial images, and building on the YOLOv7, we propose the Efficient YOLOv7-Drone. Recognizing the common presence of small objects in aerial imagery, we eliminated the less efficient P5 detection head and incorporated the P2 detection head for increased precision in small object detection. To ensure efficient feature relay from the Backbone to the Neck, channels within the CBS module were optimized. To focus the model more on the foreground and reduce redundant computations, the TGM-CESC module was introduced, achieving the generation of pixel-level constrained sparse convolution masks. Furthermore, to mitigate potential data losses from sparse convolution, we embedded the head context-enhanced method (HCEM). Comprehensive evaluation using the VisDrone and UAVDT datasets demonstrated our model’s efficacy and practical applicability. The Efficient Yolov7-Drone achieved state-of-the-art scores while ensuring real-time detection performance.
Transformer has shown its effectiveness and advantage in many computer vision tasks, for example, image classification and object re‐identification (ReID). However, existing vision transformers are stacked layer by layer, lacking direct information exchange among every layer. Inspired by DenseNet, we propose a dense transformer framework (termed Denseformer) that connects each layer to every other layer through class tokens. We demonstrate that Denseformer can consistently achieve better performance on person ReID tasks across datasets (Market‐1501, DukeMTMC, MSMT17, and Occluded‐Duke), only at a negligible increase of computation. We show that Denseformer has several compelling advantages: it pays more attention to the main parts of human bodies and obtains discriminative global features.
Multi‐focus image fusion aims to generate a clear image by fusing multiple source images. Existing deep learning‐based fusion methods often neglect the context information resulting in the loss of detail information. To address this issue, a context fusion network to merge multi‐focus images, namely CFNet, is proposed. Specifically, a context fusion module is proposed to make full use of low‐level pixels and high‐level semantic features. Particularly, the pyramid fusion mechanism and cross‐scale transfer strategy are adopted to ensure the visual and semantic consistency of the fused image. Meanwhile, to extract salient features more effectively, a spatial attention mechanism is introduced to enhance these features. Further, the pyramid loss is used to progressively refine the fused features at each scale. Experimental results show that the proposed method is superior to some existing methods in both qualitative and quantitative evaluation.
10 September 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
This paper presents a new approach to address the challenge of combining ground-based LiDAR data with satellite images for cross-view image geo-localization. The task is to figure out the position and orientation of the LiDAR within the given satellite image. While previous research has mainly focused on imagery, the integration of ground-based point clouds with satellite images has been limited due to significant differences in modalities. To release this limitation, we propose a novel method that utilizes the road structure as a consistent reference between satellite images and ground LiDAR data for accurate geo-localization. Our methodology encompasses the extraction of road structures from both point clouds and satellite images. To extract road structures from point clouds, we leverage the enhanced viewpoint beam model, which effectively captures the spatial characteristics of ground landmarks. In addition, we utilize fractional-order differential-based super-resolution technology for satellite images to improve road structure detection, ensuring reliable performance across different altitudes. Following this, our approach involves matching road structures from the ground and satellite views, simplifying the localization process to a template-matching task. Consequently, we successfully address the challenge of accurately determining the 3-DoF pose of the LiDAR within the satellite image context. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in geo-localization, outperforming comparable methods. Additionally, the approach shows versatility across various altitudes.
Road intersection is a kind of important navigation landmark, while existing detection methods exhibit clear limitations in terms of their robustness and efficiency. A real-time algorithm for road intersection detection and location in large-scale sparse point clouds is proposed in this paper. Different from traditional approaches, our method establishes the augmented viewpoints beam model to perceive the road bifurcation structure. Explicitly, the spatial features from point clouds are jointly extracted in various viewpoints in front of the robot. In addition, the evaluation metrics are designed to self-assess the quality of detection results, enabling our method to optimize the detection process in real time. Considering the scarcity of datasets for intersection detection, we also collect and annotate a VLP-16 point cloud dataset specifically for intersections, called NCP-Intersection. Quantitative and qualitative experiments demonstrate that the proposed method performs favorably against the other parallel methods. Specifically, our method performs an average precision exceeding 90% and an average processing time of approximately 88 ms/frame.