6D pose estimation from a single RGB image is a fundamental task in computer vision. In most methods of instance-level or category-level 6D pose estimation, accurate CAD models or point cloud models are indispensable part. It is not easy to quickly obtain the models of these everyday objects. To address this issue, we present a part-level object component sketch knowledge base which consists of 270 real-world object sketch models of 30 categories. Objects are disassembled into geometry components with spatial relationship according to their functions and structures, and convert them into three basic spatial structures: frustum, circular truncated cone, and sphere. We present a fast pipeline for sketch modeling with our tool. The average time for this method to build a simple model for everyday objects is about 2 minutes. Additionally, we leverage the geometric information and spatial relationships inherent in the multiple viewpoint projection maps of these sketch bases to develop a rapid inference framework for 6D pose estimation. The interpretable steps in our framework gradually retrieve and activate valid solutions in the discrete 6D pose space. Extensive experiments in real-world environments have demonstrated that our method can reliably and robustly estimate the 6D pose of objects, even without access to accurate CAD or point cloud models. Furthermore, our method achieves state-of-the-art performance, operating at a speed of 90 frames per second using parallel computing on GPU.
Robust multi-modal sensing is essential for reliable object tracking in complex environments. Although RGB-Thermal (RGBT) sensing leverages complementary spectral modalities to enhance tracking performance, existing methods often rely on static template matching, thereby neglecting the inherent temporal continuity of sensor streams. This limitation compromises tracking stability under challenging sensing conditions, such as abrupt illumination changes, severe occlusions, and thermal crossover. To address these challenges, this paper proposes TFTrack, A new type of closed-loop video stream RGBT tracking framework has adopted a deep-collaborative two-component time design: a dynamic template library, which provides spatial priors for trajectory modeling by storing the prediction box results of historical templates; and autoregressive time label propagation, which compensates for the continuity deficiency of pure template methods. Unlike traditional sparse sampling paradigms, TFTrack exploits continuous temporal informatics by integrating historical appearance features and motion trajectories within a dynamic template library, ensuring consistent perception across time. Furthermore, to maximize multi-sensor synergy, we develop an Input Feature Guide Attention Fusion (IFGAFusion) module. By dynamically recalibrating spatial importance weights under the guidance of input features, this module effectively fuses heterogeneous data streams from RGB and Thermal Infrared (TIR) sensors. Extensive qualitative and quantitative experiments demonstrate that the proposed method achieves state-of-the-art performance. Notably, TFTrack operates at 30 FPS on the LasHeR benchmark with a precision rate of 74.7% and a success rate of 59.6%, satisfying the stringent real-time and reliability requirements for practical vision-based sensing systems.
5 September 2025 in IEEE Signal Processing Letters
Video-based human-object interaction (HOI) recognition aims at labeling human and object sequences with multiple human-object interaction classes. The efficiency of existing methods still requires improvement in terms of parameter and computational complexity, which restricts the application of video-based human-object interaction recognition. In this paper, we present HOI-Mamba, a novel approach for efficient video-based human-object interaction recognition with the state space model. HOI-Mamba transforms the spatial-temporal graph to the sequence and captures the human-object interaction features with bidirectional Mamba, which leads to superior performance with higher efficiency. Experimental results on two public human-object interaction video benchmarks demonstrate that HOI-Mamba achieves significant improvements over existing methods, e.g., achieving higher F1 Score for sub-activity recognition with fewer parameters and FLOPs than existing methods both on the CAD-120 dataset and the Something-Else dataset.