The rapidly growing number of space activities is generating numerous space debris, which greatly threatens the safety of space operations. Therefore, space-based space debris surveillance is crucial for the early avoidance of spacecraft emergencies. With the progress in computer vision technology, space debris detection using optical sensors has become a promising solution. However, detecting space debris at far ranges is challenging due to its limited imaging size and unknown movement characteristics. In this paper, we propose a space debris saliency detection algorithm called SDebrisNet. The algorithm utilizes a convolutional neural network (CNN) to take into account both spatial and temporal data from sequential video images, which aim to assist in detecting small and moving space debris. Firstly, taking into account the limited resource of the space-based computational platform, a MobileNet-based space debris feature extraction structure was constructed to make the overall model more lightweight. In particular, an enhanced spatial feature module is introduced to strengthen the spatial details of small objects. Secondly, based on attention mechanisms, a constrained self-attention (CSA) module is applied to learn the spatiotemporal data from the sequential images. Finally, a space debris dataset was constructed for algorithm evaluation. The experimental results demonstrate that the method proposed in this paper is robust for detecting moving space debris with a low signal-to-noise ratio in the video. Compared to the NODAMI method, SDebrisNet shows improvements of 3.5% and 1.7% in terms of detection probability and the false alarm rate, respectively.

The COVID-19 crisis has led to an unusually large number of commercial aircraft being currently parked or stored. For airlines, airports, and civil aviation authorities around the world, monitoring, and protecting these parked aircraft to prevent them from causing human-made damage are becoming urgent problems that are receiving increasing attention. In this study, we use thermal infrared monitoring videos to establish a framework for individual surveillance around parked aircraft by proposing a human action recognition (HAR) algorithm. As the focus of this article, the proposed HAR algorithm seamlessly integrates a preprocessing module in which a novel data structure is constructed to introduce spatiotemporal information of the action; a convolutional neural network-based module for spatial feature extraction; a triple-layer convolutional long short-term memory network for temporal feature extraction; and two fully connected layers for classification. Moreover, because no infrared dataset is available for the HAR task on airport grounds at nighttime, we present a dataset called IIAR-30, which consists of eight action categories that frequently occur on airport grounds and 2000 video clips. The experimental results on the IIAR-30 dataset demonstrated that the recognition accuracy of the proposed method was higher than 96%. We also further evaluated the effectiveness of the proposed method by comparing it with five baselines and four other methods.

Tracking algorithms with low computational complexity and reliable performance are important in developing advanced driver assistance systems (DAS). This paper proposes a method of single-pedestrian tracking using thermal infrared cameras to meet the needs of DAS operating in nighttime and low-visibility conditions. The proposed algorithm uses the background-aware correlation filter (BACF) as the basic tracking framework. In order to address the problem that directly introducing the convolutional features leads to tracking performance degradation in the BACF framework, this paper proposes a fusion scheme to integrate handcrafted and convolutional features to make full use of the advantages of both the features. The proposed scheme combines response maps from convolutional and handcrafted features through fusion coefficients to improve the performance of the trackers based on the single features. In order to calculate fusion coefficients, a novel approach of searching the main peak and interference peaks of a response map is proposed by using local binary pattern values of the response map to locate all local maximum points. Experimental results show that the proposed algorithm outperforms the existing 9 competing tracking algorithms and can be used in vehicle platforms as a module of DAS to improve the safe level of driving in nighttime.

Driving in nighttime or poor illumination is much more dangerous than in daytime or rich illumination conditions so it is important to develop driver assistance systems to support safe driving in nighttime. This paper presents a pedestrian tracking algorithm using a thermal infrared camera. The proposed algorithm first computes response maps with normalized grayscale and Felzenszwalb's histogram of oriented gradient in the framework of correlation filtering. Subsequently, according to response maps, this paper proposes an approach to estimate weight ratios in fusion. The fused response map is ultimately used to complete target detection in the framework of correlation filtering. In order to test the performance of the proposed tracker, extensive comparison and tests have been performed using open-source data sets. By testing through 10 different challenges in image processing and tracking, it is evident that the performance of proposed algorithm is superior to seven representative tracking algorithms with a quite promising real-time performance.

The increasing amount of space debris in recent years has greatly threatened space operation. In order to ensure the safety level of spacecraft, space debris perception via on-orbit visual sensors has become a promising solution. However, the perception capability of visual sensors largely depends on illumination, which tends to be insufficient in dark environments. Since the images captured by visible and infrared sensors are highly complementary in dark environments, a convolutional sparse representation-based visible and infrared image fusion algorithm is proposed in this paper to expand the applicability of visual sensors. In particular, the local contrast measure is applied to obtain the refined weight map for fusing the base layers, which is more robust in a dark space environment. The algorithm can settle two significant problems in space debris surveillance, namely, improving the signal-noise ratio in a noise space environment and preserving more detailed information in a dark space environment. A space debris dataset containing registered visible and infrared images has been purposely created and used for algorithm evaluation. Experimental results demonstrate that the proposed method in this paper is effective for enhancing image qualities and can achieve favorable effects compared to other state-of-the-art algorithms.