Aircraft attitude has always been an important technical parameter during flights, especially for stages such as aircraft take-off and landing. This article presents a monocular off-board vision system for relative attitude measurement of fixed-wing aircraft. In this system, a zoom imaging subsystem is set up to track and shoot the aircraft over a large distance range. Image sequences and corresponding servo data are transmitted to the processing computer for vision algorithms. A model-based attitude tracking algorithm, which requires the computer-aided design model of the aircraft, is performed to measure the relative attitude of the aircraft with respect to the reference platform. A 2-D visual tracking algorithm and a target detection algorithm are also employed to control the vision system. The proposed attitude tracking algorithm achieves competitive results on a publicly available dataset. The full system is validated in a real-flight experiment. The results revealed that the proposed system can run in real time on 1920×1080@40 Hz videos with an accuracy better than 0.5 $^\circ$ .

A high-accuracy calibration technique using a white paper and a front coating plane mirror is proposed in this paper for line-structured light vision sensors. This method shows advantages in two aspects. First, a white paper can gain a very high-quality light stripe due to its approximate ideal diffuse Lambertian sheet, which overcomes the problems associated with the strong reflecting light and serious burrs of the light stripe on conventional rigid targets. Second, based on a front coating plane mirror with lithographic feature points, we can obtain a bilateral symmetric structure similar to a virtual binocular stereo vision to recover the 3D coordinates of the light stripe centers on white paper with high accuracy. Front coating guarantees the coplanarity with the lithographic feature points and avoids imaging distortion caused by refraction during back coating. Therefore, front coating can be used to obtain high accuracy structural parameters of the virtual binocular stereo vision sensors. Meanwhile, for the light stripe and its image in the plane mirror are auto-epipolar with all the epipolar lines arranged in parallel. These lines intersect at a vanishing point in the camera image, and this epipolar constraint is used to complete the matching of the light stripe centers without the need for the camera parameters. Experiments are conducted to demonstrate the performance of the proposed method.

Monocular 6D pose estimation for objects is an essential but challenging task that is commonly applied in computer vision and robotics. Existing two-stage methods solve for rotations with Perspective-n-Point (PnP), which still incorporates translations, resulting in accuracy degeneration. In contrast, direct regression methods adopt Convolutional Neural Networks (CNNs) to solve for rotations and translations jointly but suffer from performance gaps in rotation accuracy. In this article, we propose a novel Normal Vector guided Regression Network (NVR-Net) to directly regress the 6D pose from a single RGB image under the guidance of 3D normal vectors. Specifically, we design a novel Orientation-Aware Feature (OAF) for pose estimation. It consists of two corresponding sets of 3D normal vectors to thoroughly disentangle rotation from translation estimation. Then, we introduce a CNN to predict a dense pixelwise representation of the OAF without viewpoint ambiguity. To estimate rotations and translations individually from the OAF, we propose a novel Pose from Normal Vectors (PNV) head networks under the instruction of a differentiable closed-form solution. Finally, extensive experiments on three common benchmarks demonstrate that our approach outperforms state-of-the-art methods on rotation accuracy and removes the gap between indirect and end-to-end methods. Moreover, our method can estimate the 6D pose of a single object within an RGB image in real-time.

This article addresses the challenge of 6D aircraft pose estimation from a single RGB image during the flight. Many recent works have shown that keypoints-based approaches, which first detect keypoints and then estimate the 6D pose, achieve remarkable performance. However, it is hard to locate the keypoints precisely in complex weather scenes. In this article, we propose a novel approach, called Pose Estimation with Keypoints and Structures (PEKS), which leverages multiple intermediate representations to estimate the 6D pose. Unlike previous works, our approach simultaneously locates keypoints and structures to recover the pose parameter of aircraft through a Perspective-n-Point Structure (PnPS) algorithm. These representations integrate the local geometric information of the object and the topological relationship between components of the target, which effectively improve the accuracy and robustness of 6D pose estimation. In addition, we contribute a dataset for aircraft pose estimation which consists of 3681 real images and 216,000 rendered images. Extensive experiments on our own aircraft pose dataset and multiple open-access pose datasets (e.g., ObjectNet3D, LineMOD) demonstrate that our proposed method can accurately estimate 6D aircraft pose in various complex weather scenes while achieving the comparative performance with the state-of-the-art pose estimation methods.

Visual tracking is a crucial task in computer vision that concentrates on assessing the state of the target in each frame. Siamese based trackers possess an excellent tracking mechanism and balance the accuracy and efficiency well through continuous optimization of network structure. However, the rapid appearance variation and occlusion are still huge challenges to the accuracy and success rate of tracking task. Most Siamese based trackers rely on a fixed object template to match the target in search area and also neglect the importance of feature representation to tracking tasks. Based on the core idea of how to promote the recognition ability of trackers for the dynamic object and keep stable tracking during occlusion process, we present Siamese block attention network for online update object tracking referred to as SBAN. The proposed Siamese block attention module adopts a Siamese network structure to integrate two complementary global descriptors and establish the interdependence among channels to generate channel weights which can enhance the crucial features and restrain inessential ones. We also design an online update target module that can effectively utilize the history tracking information. The final updated target module is the integration of the given template, the process template, and the last tracking results. Experiments on four benchmarks, OTB2015, VOT2018, UAV123, LaSOT, illustrate that our tracker obtains outstanding tracking performance in accuracy and robustness while running at over 53 FPS.