Hyperspectral videos (HSVs) have more potential in object tracking than color videos thanks to their material identification ability. Nevertheless, previous works have not fully explored the benefits of the material information, resulting in limited representation ability and tracking accuracy. To address this issue, this paper introduces a material-guided multi-view fusion network for improved tracking. Specifically, we combine false-color information, hyperspectral information, and material information obtained by hyperspectral unmixing to provide a rich multi-view representation of the object. Cross-material attention is employed to capture the interaction among materials, enabling the network to focus on the most relevant materials for the target. Furthermore, leveraging the discriminative ability of material view, a novel material-guided multi-view fusion module is proposed to capture both intra-view and cross-view long-range spatial dependencies for effective feature aggregation. Thanks to the enhanced representation ability of each view and the integration of the complementary advantages of all views, our network is more capable of suppressing the tracking drift in various challenging scenes and achieving accurate object localization. Extensive experiments show that our tracker achieves state-of-the-art tracking performance. The source code will be available at https://github.com/hscv/MMF-Net.

As the basic visual morphological characteristics of molten pool, contour extraction plays an important role in on-line monitoring of welding quality. The limitations of traditional edge detection algorithms make deep learning play a more important role in the task of target segmentation. In this paper, a molten pool visual sensing system in a tungsten inert gas welding (TIG) process environment is established and the corresponding molten pool image data set is made. Based on a residual network, a multi-scale feature fusion semantic segmentation network Res-Seg is designed. In order to further improve the generalization ability of the network model, this paper uses deep convolutional generative adversarial networks (DCGAN) to supplement the molten pool data set, then performs color and morphological data enhancement before network training. By comparing with other traditional edge detection algorithms and semantic segmentation network, it is verified that the scheme has high accuracy and robustness in the actual welding environment. Moreover, a back propagation (BP) neural network is used to predict the weld width, and a fitting test is carried out for the pixel width of the molten pool and its corresponding actual weld width. The average testing error is less than 0.2 mm, which meets the welding accuracy requirements.