Multimodal image fusion is one of the important research directions in the field of multimodal fusion. This technique can realize image and data enhancement by using complementary multimodal images and be widely used in medicine, industry, security and fire protection, automatic driving and consumer electronics. In this work, we propose a transformer-based universal fusion (TUFusion) algorithm, and it has a multidomain fusion capability. The advantage of TUFusion algorithm is the design of hybrid transformer and convolutional neural network (CNN) encoder structure and a new composite attention fusion strategy, which has the ability of global and local information integration. Compared with the classical state-of-the-art multimodal image fusion methods, the experimental result on multidomain data sets showed that the TUFusion algorithm has certain universality in image fusion. Meanwhile, the TUFusion algorithm we proposed achieves good values on peak signal to noise ratio (PSNR), root mean square error (RMSE) and structural similarity index measure (SSIM). The code of the TUFusion algorithm in this article is available at https://github.com/windrunners/TUFusion.
Aiming at the problems of traditional grasping methods for mobile manipulators, such as single application scenarios, low accuracy, and complex grasping tasks, this paper proposes an object recognition grasping approach using Proximal Policy Optimization (PPO) with You Only Look Once v5 (YOLOv5), which combines a vision recognition algorithm with a deep reinforcement learning algorithm to achieve object recognition grasping. First, YOLOv5 is adopted to identify the object and obtain the location information. Second, the PPO algorithm is used for object grasping to obtain the grasping strategy. Third, the PPO algorithm is compared with the Soft Actor-Critic (SAC) and Trust Region Policy Optimization (TRPO) algorithms in batches 16 and 128, respectively. The average reward training results of the PPO, SAC, and TRPO algorithms are obtained in our work. Experimental results show that the proposed method, in terms of object recognition speed, outperforms the original YOLOv4 model. The YOLOv5 model achieves 96% precision on our own built recognition dataset, which has higher detection precision and lower hardware requirements than the YOLOv4 model. Our proposed method outperforms SAC and TRPO algorithms in object grasping, and the average reward of the PPO algorithm is improved by 93.3% and 41% compared to SAC and TRPO algorithms, respectively. Finally, through the comparison of ablation experiments, our method has the highest accuracy and mean average precision (mAP)@0.5 value of 92.3%. We demonstrate in actual physical experiments that the grasping success rate under our proposed approach reaches 100%, providing a new research strategy for object grasping by the robot manipulator.
High-quality image registration is the basis of multimodal fusion in the field of industrial welds. Due to the influence of contrast, scale, and illumination, the registration accuracy and robustness of existing algorithms are not high enough for the industrial weld field, and registration performance still needs further improvement. In this case, we propose a hybrid double branch image registration (HDBR) algorithm for multisource images, which has a parallel composite structure. The experimental result demonstrates that compared with the radiation invariant feature transformation and histogram of absolute phase consistency gradients algorithms, HDBR algorithm has higher matching accuracy and lower registration loss, and the registration performance is improved by 1.12% and 15.55%. The composite structure registration algorithm we proposed is mainly applied to the multiview weld image field, and finally high-quality multiview weld data set is obtained.
Infrared and visible image fusion is an advanced processing procedure in the measurement of visual sensors. Existing methods usually assume that the input multimodal images are clear. However, complex scenes such as smoke and haze cause significant interference to traditional fusion methods. Therefore, this paper proposes a novel unified haze removal image fusion (HRIF) framework , which achieves joint optimization of haze removal and multimodal fusion through an end-to-end network architecture. By designing a dual-channel attention mechanism, the feature extraction and noise suppression capabilities of HRIF are effectively enhanced. The model can dynamically monitor smoke distribution and adaptively adjust the fusion process. Additionally, a large multimodal fusion dataset HazeScene for smoke or haze scenarios was constructed, and an average smoke concentration (ASC) evaluation metric was proposed. Experiment results demonstrate that HRIF significantly outperforms existing state-of-the-art image fusion methods in metrics such as PSNR, MSE, CC, SSIM, and ASC, especially excelling in high-concentration haze scenario. This research provides a new solution for multimodal image fusion and visual measurement in haze environments, and has broad application prospects in multiple fields such as autonomous driving and disaster monitoring. The code of the HRIF algorithm is available at https://github.com/windrunners/HRIF .
This paper proposes a new type of three-level neutral point clamp isolated bidirectional Dc-Dc converter (3L-NPC IBDC). The proposed converter can significantly improve the power factor, and can achieve a high-voltage and high-efficiency as well. In this work, a detailed analysis the relationship between the phase and duty cycle of the transmission power and output voltage of the presented 3L-NPC IBDC converter. Furthermore, the operating range of the zero-voltage switching (ZVS) is emphatically investigated for both modes. An improved efficiency of 3L-NPC IBDC is accordingly achieved by reducing the switching loss with ZVS. A corresponding simulation model and experimental platform are then set up to demonstrate the corrections of the presented theoretical analyses, and the simulated and experimental results explicitly verify the performance predicted by the theoretical analyses.