Most reference-based image super-resolution (RefSR) methods directly leverage the raw features extracted from a pretrained VGG encoder to transfer the matched texture information from a reference image to a low-resolution image. We argue that simply operating on these raw features neglects the influence of irrelevant and redundant information and the importance of abundant high-frequency representations, leading to undesirable texture matching and transfer results. Taking the advantages of wavelet transformation, which represents the contextual and textural information of features at different scales, we propose a Wavelet-based Texture Reformation Network (WTRN) for RefSR. We first decompose the extracted texture features into low-frequency and high-frequency sub-bands and conduct feature matching on the low-frequency component. Based on the correlation map obtained from the feature matching process, we then separately swap and transfer wavelet-domain features at different stages of the network. Furthermore, a wavelet-based texture adversarial loss is proposed to make the network generate more visually plausible textures. Experiments on four benchmark datasets demonstrate that our proposed method outperforms previous RefSR methods both quantitatively and qualitatively. The source code is available at https://github.com/zskuang58/WTRN-TIP .

In this article, we propose a complete visual coverage trajectory planning framework for quadrotor unmanned aerial vehicles (UAVs) in three-dimensional (3-D) terrain environment under photogrammetric constraints. To ensure the accuracy of the 3-D reconstruction, the air-to-ground projective geometric constraints are established. To optimize exploration efficiency, a novel two-step hierarchical coverage planning algorithm is implemented. The algorithm is verified in both simulation and practical experiments. The coverage trajectory that satisfies the constraints is generated, and the 3-D reconstruction from the images collected along the way is obtained. The results show that the proposed method can improve the performance of quadrotor UAV-based 3-D visual coverage in terms of trajectory length and traversal time, as well as the quality of 3-D reconstruction.

In this paper, we address the multiple quadrotor UAVs trajectory planning optimization problem for large-scale, persistent, high-depth visual coverage tasks in three-dimensional (3-D) terrain environment. To minimize the overall energy expenditure of the UAVs for accomplishing a task, we set up an air-to-ground collaborative system which introduces base stations to hold and recharge UAVs. The system is formulated as an integer programming, and solved by a novel hierarchical reinforcement learning trajectory planning algorithm (RL-TP), in which the paths are obtained by reinforcement learning method, and then the trajectories are obtained by Bézier curve method. Both simulation and physical experiments show that RL-TP can effectively improve the efficiency and persistence of aerial visual coverage task. Note to Practitioners —While the multi-rotor UAV has been an important means for field monitoring, it suffers the problem of short battery life a lot. To make it more efficient and persistent, we use multiple UAVs and introduce ground base stations to charge the UAVs. The scenario is formulated as an air-to-ground collaborative system, and the motion planning strategy is to minimize the energy consumption. We propose a hierarchical collaborative coverage reinforcement learning trajectory planning algorithm (RL-TP) to solve it. We carry out both simulation and physical field experiments, and compare RL-TP with other popular methods. The experimental results show that the system is feasible and RL-TP performs well in both time efficiency and energy consumption. In future research, we will introduce unmanned ground vehicles to replace the stationary ground base stations to make the air-to-ground collaborative system more powerful and flexible.

The tilt-rotor unmanned aerial vehicle (TRUAV) is characterized by both multi-rotor vertical takeoff or landing and fixed-wing long-duration flight. Not only the structure of TRUAV is complex, but also the aerodynamic coupling is severe. Especially when the TRUAV is in the transition mode, the rotor flow field and wake are complicated. The abnormal variation of the aerodynamic force of the rotor blades will directly affect the balance and manipulation of the aircraft. The downwash flow generated by the rotor spiral propeller tip vortex interference will impact the wing from every direction, forming a blocked three-dimensional effect flow field on the wing surface, which seriously affects the stability of the UAV. In this paper, the transient CFD (Computational Fluid Dynamics) numerical simulation was applied to examine the flow fields of the fuselage and rotor under the transition modal of TRUAV as well as the aerodynamic disturbance. In addition, the dynamics model of the TRUAV was established based on the change of the tilt angle state, and considering the effect of rotor slip flow, we analyzed the three-dimensional flow field distribution of TRUAV in the transition mode under the aerodynamic disturbance of the fuselage and rotor, and we identified each aerodynamic parameter required for modeling. A cascade PID control strategy is designed for the TRUAV, and the results verify that the proposed TRUAV model can remain stable even when the maximum roll angle is 20 degrees. At last, the simulation results provide data support for the optimization of the TRUAV aerodynamic profile and the design of subsequent flight control methods.

Intraoperative optical navigation is widely utilized in robotic surgery systems. Typically, the observation pose of the optical tracking system (OTS) is manually adjusted and then fixed throughout the surgery. However, fixed OTS suffers from limited measurement volume (MV) and visual interferences, making consistent navigation challenging in clinics. In this paper, an operation-navigation dual-robot collaborative system is proposed for orthopedic surgeries. An extra navigation robot is introduced to actively adjust the observation pose of the OTS. A collaborative preoperative planning method is proposed for this dual-robot system, including osteotomy path planning of the operation robot and collaborative planning of the navigation robot. Firstly, osteotomy paths of the operation robot are generated according to the surgery regulations and the geometric features of the vertebral foramen. Secondly, based on the generated osteotomy paths, the collaborative planning of the navigation robot is formulated into a multi-objective optimization problem to find the optimal poses of the OTS for each osteotomy plane. Compared with fixed OTS, active navigation is capable of keeping all the targets within the MV of the OTS throughout the surgery. Semi-laminectomy on a human spine phantom is adopted as an example to experimentally evaluate the effectiveness of the proposed method. Note to Practitioners —As the demand for robot-assisted surgery is increasing, the precision of operation has become one of the key safety requirements. Preoperative planning provides guidance for the surgeon, and intraoperative navigation monitor the status of the lesion and the surgical tool in real-time. In conventional intraoperative optical navigation, the OTS is manually adjusted and remains stationary. However, the limited MV and the visual interferences introduce risks and uncertainties to the surgical system. In order to address the limitations of fixed OTS, an operation-navigation dual-robot collaborative system is proposed for orthopedic surgeries, which is composed of a surgical operation module and an active navigation module. The navigation robot is used to actively adjust the pose of the OTS. A collaborative preoperative planning for the operation-navigation dual-robot orthopedic surgery system is proposed in this paper. The effectiveness of the proposed method is verified on a human spine phantom. Experimental results show that the active navigation provides more freedom to the overall system by freely adjusting the OTS, which ensures the stability of the surgical navigation. In future work, efforts will be directed toward the identification and avoidance of the obstacle.