Robust person detection in aerial images under all-weather conditions stands as a fundamental technology pivotal to the efficacy of intelligent search and rescue (SaR) tasks. However, the challenges stem from the varied postures, sparsity, diminutiveness, and faintness of personnel objects when viewed from an air-to-ground perspective, leading to issues with insufficient feature representation and suboptimal detection accuracy. This survey commences by underscoring the extensive potential applications and the prevailing limitations associated with aerial person detection (APD) within the scope of drone-assisted SaR scenarios. To meet the requirement of APD applications, we thoroughly investigate advancements and challenges in 4 related methodologies, including object-aware methods for size and perspective variability, sample-oriented methods with sparse distribution, information-fusion methods for the issue of lighting or visibility, and lightweight methods on constrained devices. Furthermore, to foster advancements in APD, we have conducted a comprehensive APD dataset labeled as “VTSaR”, which stands out from the existing publicly accessible APD datasets by offering a greater diversity of scenes, varying personnel behaviors, flexible capture angles, differing capture heights, and an inclusion of aligned visible and infrared samples along with synthetic samples. Finally, we evaluate the performance of mainstream detection methods on VTSaR benchmarks, advocating for APD’s broader application across various domains.

Aerial Person Detection (APD) is vital for enhancing search and rescue (SaR) operations, particularly when locating victims in remote, poorly-lit areas. Despite advancements in detection technologies, achieving a balance between detection speed and accuracy on mobile devices in “edge AI” continues to pose challenges. In this paper, a lightweight distillation Network (APDNet) proposed for edge deployment of APD, which enable real-time inference as well as minimize accuracy loss during model transfer. The proposed APDNet employs a distillation network between varying-depth backbones and integrates an 8-bit quantized optimizer to reduce the floating-point operations of network parameters. Specifically, in the Teach-Assistant Distillation stage, small student models using random weight initialization are trained with pseudo-labels generated by deeper teacher models, facilitating consistent learning for a more accurate, lighter model. Moreover, a Low-Precision Quantization (LPQ) stage incorporates an offline, quantization-aware training strategy that dynamically adjusts the ranges of weight and activation function float-point values, reducing computational complexity. In order to compensate for the potential accuracy decline, a pluggable tracker updates the position and feature information of persons frame-by-frame, with tracking results integrated with detection outputs to enhance accuracy. Extensive experiments on the Heridal, Manipal-UAV, and VTSaR datasets confirm the effectiveness of APDNet, demonstrating its superior performance in edge-based aerial person detection.

Aerial-based person detection poses a significant challenge, yet it is crucial for real-world applications like airground linkage search and all-weather intelligent co-rescuing. However, existing person detection models designed for aerial images heavily rely on numerous labeled instances and exhibit limited tolerance towards complex lighting conditions commonly encountered in Search and Rescue (SaR) scenarios. This article presents the Visible-Thermal from SaR scenarios for person detection Network (VTSaRNet) to address the challenge of detecting persons situated sparsely in SaR scenes marked by intricate illumination conditions and restricted accessibility. VTSaRNet integrates the Instance Segmentation for Copy-Paste Mechanism (ISCP) using a Union Transformer Network that functions in both Visible (V) and Thermal (T) bi-modalities. Specifically, This study employs synthetic samples obtained through offline Mosaic augmentation by oversampling the local area of bulk images. Then, it utilizes the ISCP module to extract accurate boundaries of personnel instances from complex backgrounds. VTSaRNet cross-integrates the global features and encodes the correlations between two modalities through the multi-head attention module. It also adaptively recalibrates the channel responses of partial feature maps for fusion operations with the transformer module in conjunction with anchor-based detectors. Moreover, the adaptation scheme is constructed with multiple strategies to effectively handle various scenarios involving persons, and the entire network is trained end-to-end. Extensive experiments conducted on the Heridal and VTSaR datasets demonstrate the effectiveness of light-weighted VTSaRNet in achieving impressive metrics Precision of 98.3%, Recall of 96.78%, mAP@0.5 of 98.73%, and mAP@0.5:0.95 of 73.98% under self-built VTSaR dataset, respectively). This performance sets a new benchmark in person detection from aerial imagery.

Spectral Super-resolution (SR), which reconstructs high spatial-resolution hyperspectral images (HSIs) from RGB inputs, has been demonstrated to be one of the effective computational imaging techniques to acquire HSIs. Though deep neural networks have shown their superiority in such a complex mapping problem, existing networks generally involve a very complex structure with huge amounts of parameters, resulting in giant memory occupation. In this paper, a lightweight Multi-Resolution Feature Fusion Network (MRFN) is proposed, which adopts a multi-resolution feature extraction and fusion framework to fully explore RGB inputs in different scales of resolution. Specifically, a Lightweight Feature Extraction Module (LFEM), which adopts cheap convolution and attention mechanisms, is constructed to explore different scales of features under a lightweight structure. Moreover, a hybrid loss function is proposed by encountering not only pixel-value level reconstruction error, but also spectral continuity and fidelity. Experiments over three benchmark datasets, i.e., CAVE, ICVL, and NTIRE2022 datasets, have demonstrated the proposed MRFN can reconstruct HSIs from RGB inputs in higher quality with fewer parameters and computational FLOPs compared with several state-of-the-art networks.

Finding tiny persons under the drone vision was, is, and remains to be an integral and challenging task. Unmanned aerial vehicles (UAVs) with high-speed, low-altitude, and multi-perspective flight bring about violently various scales of objects, which burdens the optimization of models. Moreover, the detection performance of densely and faintly discernible person characteristics is far less than that of large objects in high-resolution aerial images. In this article, we introduce the image cropping strategy and attention mechanism based on YOLOv5 to address small person detection in the optimized VisDrone2019 dataset. Specifically, we propose a Densely Cropped and Local Attention of object detector Network (DCLANet), which is inspired by the observation that less area occupied by small objects should be fully focused and relatively magnified in the original image. DCLANet-assembled Density Map-Guided Object Detection (DMNet) in aerial images and You Only Look Twice (YOLT): Rapid Multiscale Object Detection In Satellite Imagery to crop images upon training and testing stage, meanwhile, added bottleneck attention mechanism to YOLOv5 baseline framework, which more focus on person objects other than irrelevant categories. To achieve further improvement of DCLANet, we also provide bags of useful strategies: data augmentation, label fusion, category filtering, and hyperparameter evolution. Extensive experiments on the VisDrone2019 show that DCLANet achieves state-of-the-art performanc; the detection result of person category $A P^{\text{val }}$ @0.5 is 50.04% with test-dev subset, which is substantially better than the previous SOTA method (DPNetV3) by 12.01%. In addition, on our optimized VisDrone2019 dataset, $A P^{\text{val }}$ @0.5 and $A P^{\text{test }}$ @0.5 obtained 74.95% and 62.18%, respectively. Compared to YOLOv5, DCLANet improves 3.8% or so, which is encouraging and competitive.

Spectral unmixing expresses the mixed pixels existing in hyperspectral images as the product of endmembers and their corresponding fractional abundances, which has been widely used in hyperspectral imagery analysis. However, the endmember spectra even for pixels from the same material of an image may include variability due to the influence of lighting conditions and inherent properties of materials within different pixels. Though the in situ spectral library has been used to accommodate such variability by using multiple in situ spectra to represent each kind of material, the performance improvement may be restricted due to the limited number of endmembers for each material. Therefore, in this article, spectral variability is directly extracted from an in situ endmember library and considered to be transferable among different endmembers for the first time. Furthermore, such a spectral variability is further used to augment sparse unmixing by synchronously performing endmember-based reconstruction and spectral variability-augmented reconstruction in the sparse unmixing model. By, respectively, imposing sparse and smoothness regularization over abundances and variability coefficients, a convex optimization-based spectral variability augmented sparse unmixing (SVASU) is finally proposed, and its convergence performance is also analyzed. Experiments conducted over synthetic and real-world datasets demonstrate that the proposed SVASU method not only significantly improves the unmixing performance of conventional spectral library-based unmixing but also outperforms several state-of-the-art sparse unmixing algorithms.