Large-scale flapping wing robots (FWRs) with airborne vision have important applications in visual navigation, aerial surveying, fire warning and power-line inspection. However, airborne vision and its videos suffer from strong jitters due to periodic wing flapping, which lowers the success rate of detection and measurement precision. In this paper, a robust digital video stabilization (DVS) method based on periodic jitters is proposed to provide continuous stable monitoring video without pan-tilt camera assistance. First, the periodic motion model of the FWR is established for video jitter analysis. Second, jitter frequencies in different flight states are estimated by continuous jitter acceleration. Then, feature trajectories generated from the video are adjusted adaptively for jitter frequency consistency and smoothed individually by the sampling-interpolation-averaging strategy, including the short trajectories. The stabilized video is generated by guidance from the original and smoothed trajectories. Finally, the proposed method is tested in outdoor flights with a 2.2-meter wingspan FWR and is found to outperform traditional, commercial, and deep learning DVS methods in terms of stability and robustness in various scenes and flight states.

Accurate state estimation of micro aerial vehicles (MAVs) in high-speed and dynamic environments poses a significant challenge for visual-inertial odometry (VIO) algorithms. Flapping-wing aerial vehicles (FWAVs), as an emerging flight platform, have attracted significant attention for stealth capabilities and efficient flight characteristics. Despite the availability of numerous MAV visual-inertial datasets for six-degree-of-freedom (6-DoF) state estimation, these datasets are not applicable for FWAVs with pronounced vibrations and agile maneuverability. To address this gap, we propose a large-scale flapping-wing aggressive flight visual-inertial dataset FWAF-VID. It contains diverse integrations of synchronized onboard cameras and IMU sensors. A total of 28 sequences include static flapping, calibration, and real-world flights collected in outdoor and indoor environments. The 17 flight sequences feature variations in illuminations, speeds, trajectories, camera perspectives, and flight modalities. We provide precisely aligned ground truth RTK and UWB measurements for algorithm evaluation. Furthermore, a comprehensive benchmark comparison is conducted, qualitatively and quantitatively evaluating the positioning accuracy of state-of-theart VIO algorithms. The experimental results indicate that despite the degradation in localization accuracy in high-dynamic environments, FWAF-VID can effectively support the further development of VIO algorithms for FWAVs. To the best of our knowledge, this is the first onboard benchmark dataset tailored for aggressive flight in flapping-wing airborne platforms. For the benefit of the community, all the datasets are publicly available at: https://github.com/JizhouJiang-HIT/FWAF-VID.

Bionic flapping wing robots (FWRs) with airborne vision return real-time video streams during flight for critical tasks such as aerial surveys, traffic patrols, and fire warnings. However, periodic wing flapping introduces intense jitters into the monitoring video streams. A real-time digital video stabilization (DVS) method based on inertial measurement unit (IMU) data fusion and periodic jitters, which can be operated in real time via an airborne computer and return stabilized video streams to the ground station with low latency, was proposed in this study. First, multiple continuous trajectories with equal spatial intervals were generated on the basis of airborne IMU data and optical flow tracking. Second, the lagged stable positions and velocities of the trajectories were estimated on the basis of the periodic characteristics and frequency of the flapping wings via an airborne Hall sensor. Then, the stable trajectories of the current frame were predicted with a constant-velocity model and guided to generate stable frames. Finally, outdoor flight experiments were conducted with a 1.8 m wingspan FWR. In terms of objective and subjective evaluations, the proposed method provides greater stabilization performance and robustness for different scenes and FWR flight states than existing methods do.