Fast movement of objects and illumination changes may lead to a negative effect on camera images for object detection and tracking. Event cameras are neuromorphic vision sensors that capture the vitality of a scene, mitigating data redundancy and latency. This paper proposes a new solution to moving object detection and tracking using an event frame from bio-inspired event cameras. First, an object detection method is designed using a combined event frame and a standard frame in which the detection is performed according to probability and color, respectively. Then, a detection-based object tracking method is proposed using an event frame and an improved kernel correlation filter to reduce missed detection. Further, a distance measurement method is developed using event frame-based tracking and similar triangle theory to enhance the estimation of distance between the object and camera. Experiment results demonstrate the effectiveness of the proposed methods for moving object detection and tracking.
This paper proposes a modified nonsingular fast terminal sliding mode (NFTSM) guidance law to solve the problem of ground moving target tracking for fixed-wing unmanned aerial vehicle (UAV) in a planar environment. Firstly, the loitering algorithm is analysed, which can steer the UAV to follow and circle around a ground moving target with the desired distance by heading angle control. Secondly, the effects of different parameters on the convergence time of sliding manifold is presented which is helpful for the designing of sliding manifold. Singularity can be avoided by using a modified saturation function which is obtained through a small range around the null point. Moreover, the NFTSM sliding manifold is used in the loitering algorithm. By using the Lyapunov theory, the finite-time convergence of the proposed method has been proved in the the reaching phase and the sliding phase. In order to verify the approach’s feasibility and benefits, numerical simulations are performed by using a moving target with three different motion states in comparison with the conventional sliding model control method. Simulation results indicate that, under the proposed NFTSM guidance law, the UAV can reach the desired distance in a short time.
Uncertainty of target motion, limited perception ability of onboard cameras, and constrained control have brought new challenges to unmanned aerial vehicle (UAV) dynamic target tracking control. In virtue of the powerful fitting ability and learning ability of the neural network, this paper proposes a new deep reinforcement learning (DRL)-based end-to-end control method for UAV dynamic target tracking. Firstly, a DRL-based framework using onboard camera image is established, which simplifies the traditional modularization paradigm. Secondly, neural network architecture, reward functions, and soft actor-critic (SAC)-based speed command perception algorithm are designed to train the policy network. The output of the policy network is denormalized and directly used as speed control command, which realizes the UAV dynamic target tracking. Finally, the feasibility of the proposed end-to-end control method is demonstrated by numerical simulation. The results show that the proposed DRL-based framework is feasible to simplify the traditional modularization paradigm. The UAV can track the dynamic target with rapidly changing of speed and direction.
To achieve the perception-based autonomous control of UAVs, schemes with onboard sensing and computing are popular in state-of-the-art work, which often consist of several separated modules with respective complicated algorithms. Most methods depend on handcrafted designs and prior models with little capacity for adaptation and generalization. Inspired by the research on deep reinforcement learning, this paper proposes a new end-to-end autonomous control method to simplify the separate modules in the traditional control pipeline into a single neural network. An image-based reinforcement learning framework is established, depending on the design of the network architecture and the reward function. Training is performed with model-free algorithms developed according to the specific mission, and the control policy network can map the input image directly to the continuous actuator control command. A simulation environment for the scenario of UAV landing was built. In addition, the results under different typical cases, including both the small and large initial lateral or heading angle offsets, show that the proposed end-to-end method is feasible for perception-based autonomous control.
Visual-inertial odometry is critical for Unmanned Aerial Vehicles (UAVs) and robotics. However, there are problems of motion drift and motion blur in sharp brightness changes and fast-motion scenes. It may cause the degradation of image quality, which leads to poor location. Event cameras are bio-inspired vision sensors that offer significant advantages in high-dynamic scenes. Leveraging this property, this paper presents a new range and event-based visual-inertial odometry (REVIO). Firstly, we propose an event-based visual-inertial odometry (EVIO) using sliding window nonlinear optimization. Secondly, REVIO is developed on the basis of EVIO, which fuses events and distances to obtain clear event images and improves the accuracy of position estimation by constructing additional range constraints. Finally, the EVIO and REVIO are tested in three experiments—dataset, handheld and flight—to evaluate the localization performance. The error of REVIO can be reduced by nearly 29% compared with EVIO in the handheld experiment and almost 28% compared with VINS-Mono in the flight experiment, which demonstrates the higher accuracy of REVIO in some fast-motion and high-dynamic scenes.
This paper proposes a unified attitude controller based on the modified linear active disturbance rejection control (LADRC) for a dual-tiltrotor unmanned aerial vehicle (UAV) with cyclic pitch to achieve accurate attitude control despite its nonlinear and time-varying characteristics during flight mode transitions. The proposed control algorithm has higher robustness against model mismatch compared with the model-based control algorithms. The modified LADRC utilizes the state feedbacks from the onboard sensors like IMU and Pitot tube instead of the mathematical model of the plane. It has less dependency on the accurate dynamics model of the dual-tiltrotor UAV, which can hardly be built. In contrast to the original LADRC, an actuator model is integrated into the modified LADRC to compensate for the non-negligible slow rotor flapping dynamics and servo dynamics. This modification eliminates the oscillation of the original LADRC when applied on the plant with slow-response actuators, such as propeller and rotors of the helicopter. In this way, the stability and performance of the controller are improved. The controller replaces the gain-scheduling or the control logic switching by a unified controller structure, which simplifies the design approach of the controller for different flight modes. The effectiveness of the modified LADRC and the performance of the unified attitude controller are demonstrated in both simulation and flight tests using a dual-tiltrotor UAV. The attitude control error is less than ±4° during the conversion flight. The control rising time in different flight modes is all about 0.5 s, despite the variations in the airspeed and tilt angle. The flight results show that the controller guarantees high control accuracy and uniform control quality in different flight modes.