Unmanned aerial vehicle (UAV) object detection plays a crucial role in civil, commercial, and military domains. However, the high proportion of small objects in UAV images and the limited platform resources lead to the low accuracy of most of the existing detection models embedded in UAVs, and it is difficult to strike a good balance between detection performance and resource consumption. To alleviate the above problems, we optimize YOLOv8 and propose an object detection model based on UAV aerial photography scenarios, called UAV-YOLOv8. Firstly, Wise-IoU (WIoU) v3 is used as a bounding box regression loss, and a wise gradient allocation strategy makes the model focus more on common-quality samples, thus improving the localization ability of the model. Secondly, an attention mechanism called BiFormer is introduced to optimize the backbone network, which improves the model’s attention to critical information. Finally, we design a feature processing module named Focal FasterNet block (FFNB) and propose two new detection scales based on this module, which makes the shallow features and deep features fully integrated. The proposed multiscale feature fusion network substantially increased the detection performance of the model and reduces the missed detection rate of small objects. The experimental results show that our model has fewer parameters compared to the baseline model and has a mean detection accuracy higher than the baseline model by 7.7%. Compared with other mainstream models, the overall performance of our model is much better. The proposed method effectively improves the ability to detect small objects. There is room to optimize the detection effectiveness of our model for small and feature-less objects (such as bicycle-type vehicles), as we will address in subsequent research.

Calibrating the extrinsic parameters on a system of 3D Light Detection And Ranging (LiDAR) and the monocular camera is a challenging task, because accurate 3D-2D or 3D-3D point correspondences are hard to establish from the sparse LiDAR point clouds in the calibration procedure. In this paper, we propose a geometric calibration method for estimating the extrinsic parameters of the LiDAR-camera system. In this method, a novel combination of planar boards with chessboard patterns and auxiliary calibration objects are proposed. The planar chessboard provides 3D-2D and 3D-3D point correspondences. Auxiliary calibration objects provide extra constraints for stable calibration results. After that, a novel geometric optimization framework is proposed to utilize these point correspondences, thus leading calibration results robust to LiDAR sensor noise. Besides, we contribute an automatic approach to extract point clouds of calibration objects. In the experiments, our method has a superior performance over state-of-the-art calibration methods. Furthermore, we verify our method by computing depth map and improvements can also be found. These results demonstrate that our method performance on the LiDAR-camera system is applicable for future advanced visual applications.

The frequent illegal use of drones poses a serious threat to public security and property. Counter-drones are crucial tools. The prerequisite for an effective counter-drone is to detect drones accurately. With the rapid advancements in computer vision, vision-based drone detection methods have emerged as a hot topic of research. However, current reviews of vision-based drone detection are less focused on algorithmic summarization and analysis. For this reason, this survey aims to comprehensively review the latest methods for vision-based drone detection in complex environments, with the goal of providing a more thorough reference and guidance for related research. We first explore the imaging and drone characteristics in complex environments and summarize the main challenges of visual UAV detection. Then, we summarize the existing solutions for the main challenges. Finally, we systematically organize and introduce the commonly used datasets and evaluation metrics and conduct experiment comparisons based on the representative methods. We not only reveal the current development status of visual UAV detection but also analyze the deficiencies in current research. On this basis, we further look forward to future research directions and possible breakthroughs, with a view to providing useful insights for further research and development in related fields.

Voxel is one of the common structural representation of three dimensional (3D) point cloud. Due to the sparsity of point cloud generated by light detection and ranging (LiDAR), there is the extreme imbalance in the foreground and background voxels. It decreases the accuracy of 3D object detection, has the negative effect on intelligent driving safety. To overcome this problem, we present a saliency prediction based 3D object detector SP-Det in this paper. Although foreground voxels have the sufficient feature of object, it is difficult to localize the foreground region from voxel space with the larger background region. We design an auxiliary learning task, saliency prediction (SP). It benefits 3D detector in identifying the foreground region. SP task uses label diffusion to alleviate the label imbalance. It reduces the learning difficulty of saliency in voxel and bird's eye view (BEV) spaces. After that, to strengthen feature interaction from the sparse foreground region, we design saliency fusion (SF) module to fuse the learning result in SP task. It utilizes voxel and BEV saliency maps as progressive attention to resist the redundant feature from background region. To aggregate more foreground feature inside 3D and BEV region of interest (RoI), we design hybrid grid maps based RoI pooling (Hybrid-RoI pooling). Experiments are conducted in STF dataset. The adverse weather enlarges the sparsity of LiDAR point cloud, increasing the difficulty of object detection. SP-Det identifies and leverages foreground region, and achieves the performance better than the current methods. Hence, we believe that SP-Det benefits to LiDAR based 3D scene understanding in the adverse weather.

A system with light detection and ranging (LiDAR) and camera (named as LiDAR-camera system) plays the essential role in intelligent vehicle (IV), for it provides 3D spatial and 2D texture features for 3D scene understanding. To leverage LiDAR point cloud and image, extrinsic calibration is a crucial technique, for it can align 2D pixel and 3D point in the pixel-level accuracy. With the rapid development of IV, calibration demand is shifted from offline to online, from the specific scenes to the open scenes. It brings new challenge to the calibration task. Although numbers of approaches have been proposed in the last decade, there lacks an in-depth summary about this topic. Thus, we conduct a survey of extrinsic calibration. Theoretically, the key of calibration is to build correspondence from LiDAR point cloud and optical image. From the viewpoint of correspondence, we attempt to divide the mainstream approaches into explicit and implicit correspondence based methods. After that, we summarize both the strength and weakness of the current works, provide the methods comparison, and list the open-source implementations. Finally, we analyze the tendency of calibration approach, discuss the remained problems in this field. We believe that this survey benefits to the community of autonomous driving.

Light detection and ranging (LiDAR) is an essential sensor for three dimensional (3D) object detection via generating 3D point cloud of the surroundings, and it has been widely used in the various visual applications, especially autonomous driving. However, limited numbers of labeled LiDAR datasets brutally restrain the development of 3D object detector, and this situation breeds an urgent demand on data augmentation in this field. By far, most of the traditional methods reuse the labeled samples, while those unlabeled are hastily untaken. Motivated by this, we propose a Realistic Simulator based data augmentation (RS-Aug). It aims to construct augmented real scenes to enrich the diversity of training dataset. To train 3D object detector in a supervised learning way, the first step of RS-Aug is auto-annotation. Time-continuous LiDAR frames are used to construct the dense scene, which is beneficial to annotation and the subsequent rendering augmentation. However, 3D points with incorrect semantic labels are naturally gathered during multi-view reconstruction, causing the negative effect on auto-annotation. We propose an algorithm of cluster guided $k$ -nearest neighbor (c- $k$ NN). It emphasizes on de-nosing semantic labels of clustered points using distance and intensity constraints. Then, the next step of RS-Aug is rendering augmentation on the real scene. To enhance the rendering quality using collision and distance constraints with the less computation complexity, we propose a scheme of heuristic search (HS) based object insertion. It estimates the proper position of the inserted object from 2D bird’s eye view (BEV). Experiments demonstrate the de-noising accuracy of c- $k$ NN, rendering quality of HS based object insertion, and improvement of RS-Aug on object detection.