18 December 2025 in IEEE Transactions on Multimedia
Kinesthetic and tactile information can represent the physical states of objects, encompassing roughness, stiffness, motion, force, and other attributes. The introduction of these can enhance imprecise recognition that relies solely on visual information in cases of light disturbance, occlusion and camouflage. Nevertheless, this task is still challenging due to the heterogeneity among visual, tactile and kinesthetic data. To address this issue, this paper delves into the alignment of heterogeneous data dimensions, the fusion of heterogeneous data features, and the optimization of learning rates for multi-source heterogeneous sensor learning models. Consequently, an effective Visual-Kinesthetic-Tactile Information Fusion (VikitaFusion) network is proposed, which comprises: 1) heterogeneous data extractors that align visual images with tactile and kinesthetic data through image-to-sequence projection; 2) a visual-kinesthetic-tactile Transformer-based domain fusion that mimics human multi-sensory fusion perception through a feature-level fusion block and dynamic fusion blocks; 3) a Periodic Triangulation Learning Rate (PTLR) method aimed at optimizing the learning rate for performance enhancement in multi-source heterogeneous sensor learning models. Extensive experiments demonstrate that VikitaFusion outperforms current state-of-the-art methods with higher recognition accuracy and a lower parameter size. Source code will be made publicly available after the peer review process.
11 February 2026 in TechRxiv
Visual-tactile perception is fundamental to enabling intelligent robots to recognize and adaptively interact with dynamic environments. However, the development of multimodal perception algorithms faces a key challenge: the lack of largescale, standardized datasets required for robust optimization, which is exacerbated by the wide variation in existing datasets' scale, quality, and domain specificity. This paper surveys the field of visual-tactile perception in robotics, focusing on three core tasks and evaluates the performance of integrated algorithmic approaches. To address the dataset gap, we systematically review and categorize 21 publicly available visual-tactile datasets, dividing them into numerical (e.g., LMT, PHAC-2) and image-based (e.g., GelFabric) types. We analyze their collection methodologies, structural composition, and application contexts, comparing the strengths and limitations of each type. Finally, we discuss future development trends in dataset creation and multimodal perception. This work serves as a foundational reference for visualtactile perception research and provides the first comprehensive survey dedicated to visual-tactile datasets in intelligent robotics, aiming to guide future data-driven research and algorithmic design.