Image segmentation, which has become a research hotspot in the field of image processing and computer vision, refers to the process of dividing an image into meaningful and non-overlapping regions, and it is an essential step in natural scene understanding. Despite decades of effort and many achievements, there are still challenges in feature extraction and model design. In this paper, we review the advancement in image segmentation methods systematically. According to the segmentation principles and image data characteristics, three important stages of image segmentation are mainly reviewed, which are classic segmentation, collaborative segmentation, and semantic segmentation based on deep learning. We elaborate on the main algorithms and key techniques in each stage, compare, and summarize the advantages and defects of different segmentation models, and discuss their applicability. Finally, we analyze the main challenges and development trends of image segmentation techniques.

Efficiently and accurately separating infrared (IR) small targets from complex backgrounds presents a significant challenge. Numerous studies in the literature have proposed various feature fusion modules designed specifically to enhance the extraction of IR small target features. While these designs offer some incremental improvement to the accuracy of IR small target detection, they come at a steep cost of significantly increasing network parameters and FLOPs. Striving for a balance between computational efficiency and model accuracy, we decided to forgo these complex feature fusion modules. Instead, we developed a new lightweight encoding and decoding structure known as the Lightweight IR Small Target Segmentation Network (LW-IRSTNet). This structure integrates regular convolutions, depthwise separable convolutions, atrous convolutions, and asymmetric convolutions modules. In addition, we devised post-processing modules including an eight-neighborhood clustering algorithm and an online target feature adjustment strategy. Experimental results indicate that: 1) the segmentation accuracy metrics of LW-IRSTNet match the best results of 14 state-of-the-art comparative baselines; 2) the parameters and FLOPs of LW-IRSTNet, at only 0.16M and 303M respectively, are significantly smaller in comparison to these baselines; and 3) the post-processing modules enhance both user-friendliness and the robustness of algorithm deployment. Moreover, LW-IRSTNet has been successfully implemented on both embedded platforms and websites, expanding its range of applications. Utilizing the ONNX framework, NPU acceleration, and CPU multi-threaded resource allocation, we have been able to achieve high-performance inference capabilities, as well as online dynamic threshold adjustment with the LW-IRSTNet. The source codes for this project can be accessed at https://github.com/kourenke/LW-IRSTNet.

Arbitrary-oriented object detection is vital for improving UAV sensing and has promising applications. However, challenges persist in detecting objects under extreme conditions like low-illumination and strong occlusion. Cross-modal feature fusion enhances detection in complex environments but current methods do not adequately learn the features of each modality for the current environment, resulting in degraded performance. To tackle this, we propose the CRSIOD network that effectively learns diverse sensor image features to capture distinct scenarios and target characteristics. Firstly, we design an illumination perception module to guide the object detection network in performing various feature processing tasks. Secondly, to leverage the respective advantages of two modalities and mitigate their negative impacts, we introduce an uncertainty aware module to quantify the uncertainties present in each modality as weights to motivate the network to learn in a direction favorable for optimal object detection. Moreover, in the object detection network, we design a two-stream backbone network based on the attention mechanism to enhance the learning of difficult samples, utilize the CMAFF module to fully extract the shared and complementary features between the two modalities, and design a three-branch feature enhancement network to enhance the learning of the three modal features separately. Finally, to optimize detection results, we design light perception non-maximum suppression and improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method CRSIOD on the Drone Vehicle dataset of public UAV aerial images. Compared with the existing commonly used methods, CRSIOD achieves state-of-the-art detection performance.

To solve the problem of infrared (IR) small target tracking loss or error caused by factors such as scale changes, motion blur, occlusion, etc., this paper proposes a multi-strategy fusion tracking algorithm using an IR small target segmentation network as the detection head, which mainly includes six strategies: target pixel clustering, target feature threshold adjustment, large area search, small area tracking, gate tracking, and coordinate solution. First, candidate targets are obtained through the IR small target segmentation network and pixel clustering strategy. Second, the range of candidate targets is further reduced through threshold adjustment strategies. Then, real-time tracking of IR small targets is achieved through large area search, small area tracking, and wave gate tracking strategies. Finally, the longitude, latitude, and altitude of the tracked target are obtained through coordinate calculation strategies. Both qualitative and quantitative experiments based on real IR small target sequences verify that our algorithm can achieve more satisfactory performances in terms of success rate, precision, and robustness compared with other typical visual trackers. In addition, we have deployed tracking algorithms on the Orange Pi 5 embedded platform, and the tracking speed meets the real-time requirements.

Effective detection of small targets plays a pivotal role in infrared (IR) search and track applications for modern military defense or attack. However, IR small targets are very difficult to detect because of their weak brightness, small size, and lack of shape, structure, texture, and other information elements. In order to simultaneously satisfy the robustness and timeliness of target detection, inspired by density peak clustering and the human visual system, an idea combining an improved density peak global search and local contrast calculation is proposed. First, the positions of candidate targets are determined in the preprocessed image using the improved density peak global search method (IDPGSM). Second, the saliency map is obtained using the double-weights enhanced local contrast method (DWELCM) for the candidate target neighborhood. Finally, adaptive threshold segmentation is used to detect IR small targets. Through comprehensive analysis of five evaluation indicators, the experimental results on seven real sequences and three hundred IR images of different scenes that the proposed method has better detection performance compared with six baseline methods. It can quickly and accurately determine the small target position in the case of severe background clutter and noise interference.

With the rapid advancements in deep learning technology, various deep learning-based object detection algorithms have found extensive applications in UAV-related tasks. However, motivated by the fact that current object detection algorithms for unimodal aerial remote sensing images (ARSI) fail to achieve around-the-clock object detection. To tackle this, we propose an around-the-clock object detection algorithm YOLOFIV that fuses infrared and visible features. First, we design a dual-stream backbone network based on the attention mechanism to adequately extract the features of both modalities. Moreover, the ECA attention mechanism is integrated into the feature enhancement network to amplify attention toward challenging detection scenarios. Finally, we improve the horizontal detection head to a rotating one to preserve object orientation. We evaluate the proposed method YOLOFIV on the widely used Drone Vehicle dataset, YOLOFIV achieves an accuracy of 64.71% (in terms of $mAP_{0.5}$ ), accuracy improvement of 8.32% over baseline bimodal model, similar performance to UACMD designed for ARSI object detection but with 92.35% reduction in parameter count, and 17.87 times speedup. The results show that our approach achieves round-the-clock object detection while maintaining a favorable accuracy-speed trade-off.