Synthetic aperture radar (SAR) satellites produce large quantities of remote sensing images that are unaffected by weather conditions and, therefore, widely used in marine surveillance. However, because of the hysteresis of satellite-ground communication and the massive quantity of remote sensing images, rapid analysis is not possible and real-time information for emergency situations is restricted. To solve this problem, this paper proposes an on-board ship detection scheme that is based on the traditional constant false alarm rate (CFAR) method and lightweight deep learning. This scheme can be used by the SAR satellite on-board computing platform to achieve near real-time image processing and data transmission. First, we use CFAR to conduct the initial ship detection and then apply the You Only Look Once version 4 (YOLOv4) method to obtain more accurate final results. We built a ground verification system to assess the feasibility of our scheme. With the help of the embedded Graphic Processing Unit (GPU) with high integration, our method achieved 85.9% precision for the experimental data, and the experimental results showed that the processing time was nearly half that required by traditional methods.
In recent years, satellites capable of capturing videos have been developed and launched to provide high definition satellite videos that enable applications far beyond the capabilities of remotely sensed imagery. Moving object detection and moving object tracking are among the most essential and challenging tasks, but existing studies have mainly focused on vehicles. To accurately detect and then track more complex moving objects, specifically airplanes, we need to address the challenges posed by the new data. First, slow-moving airplanes may cause foreground aperture problem during detection. Second, various disturbances, especially parallax motion, may cause false detection. Third, airplanes may perform complex motions, which requires a rotation-invariant and scale-invariant tracking algorithm. To tackle these difficulties, we first develop an Improved Gaussian-based Background Subtractor (IPGBBS) algorithm for moving airplane detection. This algorithm adopts a novel strategy for background and foreground adaptation, which can effectively deal with the foreground aperture problem. Then, the detected moving airplanes are tracked by a Primary Scale Invariant Feature Transform (P-SIFT) keypoint matching algorithm. The P-SIFT keypoint of an airplane exhibits high distinctiveness and repeatability. More importantly, it provides a highly rotation-invariant and scale-invariant feature vector that can be used in the matching process to determine the new locations of the airplane in the frame sequence. The method was tested on a satellite video with eight moving airplanes. Compared with state-of-the-art algorithms, our IPGBBS algorithm achieved the best detection accuracy with the highest F1 score of 0.94 and also demonstrated its superiority on parallax motion suppression. The P-SIFT keypoint matching algorithm could successfully track seven out of the eight airplanes. Based on the tracking results, movement trajectories of the airplanes and their dynamic properties were also estimated.
The emerging satellite videos provide the opportunity to detect moving objects and track their trajectories, which were not possible for remotely sensed imagery with limited temporal resolution. So far, most studies using satellite video data have been concentrated on traffic monitoring through detecting and tracking moving cars, whereas the studies on other moving objects such as airplanes are limited. In this paper, an integrated method for monitoring moving airplanes from a satellite video is proposed. First, we design a normalized frame difference labeling (NFDL) algorithm to detect moving airplanes, which adopts a non-recursive strategy to deliver stable detection throughout the whole video. Second, the template matching (TM) technique is utilized for tracking the detected moving airplanes in the frame sequence by improved similarity measures (ISMs) with better rotation invariance and model drift suppression ability. Template matching with improved similarity measures (TM-ISMs) is further implemented to handle the leave-the-scene problem. The developed method is tested on a satellite video to detect and track eleven moving airplanes. Our NFDL algorithm successfully detects all the moving airplanes with the highest F1 score of 0.88 among existing algorithms. The performance of TM-ISMs is compared with both its traditional counterparts and other state-of-the-art tracking algorithms. The experimental results show that TM-ISMs can handle both rotation and leave-the-scene problems. Moreover, TM-ISMs achieve a very high tracking accuracy of 0.921 and the highest tracking speed of 470.62 frames per second.
To realize the application of super-resolution technology from theory to practice, and to improve microsatellite spatial resolution, we propose a special super-resolution algorithm based on the multi-modality super-CMOS sensor which can adapt to the limited operation capacity of microsatellite computers. First, we designed an oblique sampling mode with the sensor rotated at an angle of to obtain high overlap ratio images with sub-pixel displacement. Secondly, the proposed super-resolution algorithm was applied to reconstruct the final high-resolution image. Because the satellite equipped with this sensor is scheduled to be launched this year, we also designed the simulation mode of conventional sampling and the oblique sampling of the sensor to obtain the comparison and experimental data. Lastly, we evaluated the super-resolution quality of images, the effectiveness, the practicality, and the efficiency of the algorithm. The results of the experiments showed that the satellite-using super-resolution algorithm combined with multi-modality super-CMOS sensor oblique-mode sampling can increase the spatial resolution of an image by about 2 times. The algorithm is simple and highly efficient, and can realize the super-resolution reconstruction of two remote-sensing images within 0.713 s, which has good performance on the microsatellite.
18 March 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
On-board processing has become an inevitable choice for the development of remote sensing satellites. For the satellites with dual-line array asynchronous push-broom mode, image registration is crucial for pixel localization and alignment between dual-band strip data. However, due to the high computational complexity of feature detection and feature extraction, it is still a challenge to realize real-time registration for on-board edge devices with limited computing power and power consumption. Therefore, this paper proposes a hardwareoptimized architecture with high-performance and low-energy for on-board registration based on “ARM+FPGA”. On one hand, optimized methods for the image registration algorithm are proposed from both software and hardware perspectives, providing a solution for on-board registration. On the other hand, in the face of the current interface environment of satellites, a design scheme for versatile hardware architecture is proposed. It provides a foundational technical route for hardware deployment of on-board processing. Experimental results show that in this hardware-accelerated architecture, the average acceleration effect of registration algorithm can reach 15 times compared to the unoptimized version. Additionally, the average power consumption is reduced by 60%, and the hardware resource utilization is less than 40%. Importantly, the algorithm's accuracy remains unaffected by these optimizations. The onboard intelligent processing payload deployed with this hardware architecture has been successfully launched and validated in February 2022 in the orbit of the MN200Sar-1 satellite from China. It improves the real-time capability for on-board processing, and aims to achieve a new imaging mechanism where target and information transmission replace strip image transmission.
The high-precision geometric positioning of optical remote sensing satellites is the prerequisite to determine the application capability of satellite image products. Its positioning accuracy is related to the observation accuracy of each link in the imaging process, including satellite attitude, orbit measurement accuracy, time synchronization accuracy, camera measurement accuracy, and so on. Untimely and inaccurate on-orbit calibration will lead to great geometric positioning errors. To optimize the positioning accuracy of satellite images with the rational function model (RFM) under low positioning accuracy, our paper proposes an improved geometric quality model based on the reorientation of internal and external orientation elements in the RFM model of remote sensing images. By establishing the rational function positioning model, the external orientation model, and the internal orientation model, the original image can be reorientated. Then, we use the improved model to generate uniformly distributed virtual ground control points. By analyzing and verifying the relationship between each rational polynomial coefficient (RPC) and its influence on geometric positioning accuracy, we propose an RPC coefficients optimization method based on image offset correction and positioning dominant coefficients. Finally, we use the small satellite “MN200Sar-1” with low geometric accuracy for experimental verification. The results show that the model can effectively eliminate the errors of internal and external elements in the on-orbit calibration, and the positioning accuracy is improved from one hundred pixels to one pixel. At the same time, the rational polynomial dominant coefficient optimization method can improve geometric positioning accuracy without introducing additional compensation parameters.