SWIR imaging bears considerable advantages over visible-light (color) and thermal images in certain challenging propagation conditions. Thus, the SWIR imaging channel is frequently used in multi-spectral imaging systems (MSIS) for long-range surveillance in combination with color and thermal imaging to improve the probability of correct operation in various day, night and climate conditions. Integration of deep-learning (DL)-based real-time object detection in MSIS enables an increase in efficient utilization for complex long-range surveillance solutions such as border or critical assets control. Unfortunately, a lack of datasets for DL-based object detection models training for the SWIR channel limits their performance. To overcome this, by using the MSIS setting we propose a new cross-spectral automatic data annotation methodology for SWIR channel training dataset creation, in which the visible-light channel provides a source for detecting object types and bounding boxes which are then transformed to the SWIR channel. A mathematical image transformation that overcomes differences between the SWIR and color channel and their image distortion effects for various magnifications are explained in detail. With the proposed cross-spectral methodology, the goal of the paper is to improve object detection in SWIR images captured in challenging outdoor scenes. Experimental tests for two object types (cars and persons) using a state-of-the-art YOLOX model demonstrate that retraining with the proposed automatic cross-spectrally created SWIR image dataset significantly improves average detection precision. We achieved excellent improvements in detection performance in various variants of the YOLOX model (nano, tiny and x).

Short-wave infrared (SWIR) imaging has significant advantages in challenging propagation conditions where the effectiveness of visible-light and thermal imaging is limited. Object tracking in SWIR imaging is particularly difficult due to lack of color information, but also because of occlusions and maneuvers of the tracked object. This paper proposes a new algorithm for object tracking in SWIR imaging, using a kernelized correlation filter (KCF) as a basic tracker. To overcome occlusions, the paper proposes the use of the Kalman filter as a predictor and a method to expand the object search area. Expanding the object search area helps in better re-detection of the object after occlusion, but also leads to the occasional appearance of errors in measurement data that can lead to object loss. These errors can be treated as outliers. To cope with outliers, Huber’s M-robust approach is applied, so this paper proposes robustification of the Kalman filter by introducing a nonlinear Huber’s influence function in the Kalman filter estimation step. However, robustness to outliers comes at the cost of reduced estimator efficiency. To make a balance between desired estimator efficiency and resistance to outliers, a new adaptive M-robustified Kalman filter is proposed. This is achieved by adjusting the saturation threshold of the influence function using the detection confidence information from the basic KCF tracker. Experimental results on the created dataset of SWIR video sequences indicate that the proposed algorithm achieves a better performance than state-of-the-art trackers in tracking the maneuvering object in the presence of occlusions.

Face detection systems with color cameras were rapidly evolving and have been well researched. In environments with good visibility they can reach excellent accuracy. But changes in illumination conditions can result in performance degradation, which is the one of the major limitations in visible light face detection systems. The solution to this problem could be in using thermal infrared cameras, since their operation doesn't depend on illumination. Recent studies have shown that deep learning methods can achieve an impressive performance on object detection tasks, and face detection in particular. The goal of this paper is to find an effective way to take advantages from thermal infrared spectra and provide an analysis of various image degradation influence on thermal face detection performance in a system based on R-CNN with special accent on implementation on a hardware platform for video signal processing that institute Vlatacom has developed, called vVSP.

Having high precision ground-truth data is a very important factor for the development and evaluation of computer vision algorithms such as digital video stabilization. However, genera ...

This paper presents an analysis comparing theoretical model predictions and experimental measurements of the number of pixels on target (PoT) for unmanned aerial vehicles (UAVs) using multi-sensor electro-optical and infrared (EO/IR) systems. The primary objective is to evaluate the fidelity of theoretical range prediction models that use the number of PoT as a metric, comparing these calculated values against experimental field measurements. The data used in this analysis were acquired using both visible (VIS) and mid-wave infrared (MWIR) channels, with the target UAV positioned at known reference distances up to 900 m. The results quantitatively demonstrate the deviations between the two methodologies, providing critical insights into the practical limitations of performance prediction models for early drone detection systems operating under field conditions.

A new class of modified minimum state error variance (MSEV) state-space based optimal linear quadratic Gaussian (LQG) regulators for closed-loop structures with estimated feedback has been proposed in this article. The negative feedback path is designed as the cascade of the digital LQG regulator and discrete Kalman state observer. The proposed design enables tracking of a time-varying reference input using the predictive control approach. Moreover, the proposed tracking method utilizes a multivariable continuous-time Cauchy state-space model of nonlinear, nonstationary dynamic systems. The resulting control strategy is approximately optimal, as the optimality of the LQG design holds locally for each linearized model around the respective operating point and does not extend to the global nonlinear system. In this sense, starting from the prespecified nominal state trajectory to be tracked, a numerical optimization procedure minimizing the squared tracking error at each step by using the Nelder–Mead direct search simplex algorithm under the required constraints on the input signal has been developed. The LQG regulator and Kalman state observer are designed by utilizing the linear discrete-time state variable models that properly approximate the nonlinear system dynamics across the nominal state trajectory. The performance of the proposed design is validated by simulating a six-degree-of-freedom nonlinear aircraft model across typical flight regimes.