26 January 2026 in IEEE Access
Text–Video Retrieval (TVR) retrieves videos that match a natural language query, but extending image–text models such as CLIP to videos is fundamentally limited by the lack of temporal modeling. Videos exhibit frame-wise heterogeneity in appearance and motion, and compressing all frames into a single representation often obscures temporal structure and semantic transitions. To address this, we propose Temporal-Aware Mixture-of-Experts for Text-Video Retrieval (TAME), a CLIP-based framework that jointly models frame-level structure and temporal relations. First, we integrate sparse Mixture-of-Experts (MoE) layers into both CLIP encoders and apply frame-consistent routing on the vision branch so that experts specialize according to frame-level visual patterns while preserving the original vision–language alignment. Second, we introduce Frame–Temporal (FT) tokens that aggregate global cross-frame information and feed it back to each frame, enabling the visual encoder to capture long-range temporal dependencies without harming local details. Third, we design a Cross-Temporal Interaction and Aggregation (CTIA) module that refines frame-wise sentence–video similarities through staged temporal filtering and fusion. Experiments on standard TVR benchmarks show that TAME consistently improves over CLIP-based baselines. On MSR-VTT, it yields a +4.0 R@1 improvement over CLIP4Clip, and also achieves consistent gains on DiDeMo, MSVD, LSMDC and ActivityNet. The code is available at https://github.com/sejong-rcv/TAME.
1 May 2025 in Journal of Institute of Control, Robotics and Systems
Pedestrian detection has long been a prominent area of focus in computer vision due to its extensive applications, particularly in the development of autonomous driving systems. While color image-based pedestrian detection has achieved significant progress, its vulnerability to illumination changes has prompted more research into multispectral pedestrian detection, which leverages both visible and infrared modalities for robust detection in varying lighting conditions. Despite advances in multispectral detection models, research on applying self-supervised learning for feature representation in this area remains limited. In this study, we extend existing self-supervised learning approaches for color image data to multispectral pedestrian detection. By pre-training with self-supervised learning methods and fine-tuning, we evaluate their extensibility to the multispectral domain. Experimental results demonstrate that self-supervised learning enhances multispectral pedestrian detection, especially in nighttime conditions, outperforming models pretrained only on color modalities. Furthermore, we analyze the framework of self-supervised learning method that perform well on multispectral data, providing insights into which framework elements enhance multispectral pedestrian detection. This study underscores the potential of self-supervised learning in multispectral pedestrian detection and its role in advancing detection robustness for critical applications such as autonomous driving.
27 April 2026 in SSRN Electronic Journal
Recent advances in vision-language models (VLMs), have expanded the scope of scene graph generation by enabling open-world visual relation detection. Nevertheless, VLMs still suffer from inherent limitations in spatial reasoning, particularly in modeling spatial configurations and attributive cues. Prior studies on open-vocabulary scene graph generation (OV-SGG) have tended to rely on adjustments of the VLM backbone, while leaving the fundamental challenges of the task largely unaddressed. We argue that the core bottleneck of OV-SGG lies in the combinatorial explosion of possible \textless subject-predicate-object\textgreater \, triplets, combined with the need for adaptability to novel environments. To overcome this issue, we introduce a hyperbolic geometry based framework, exploiting its exponential expansion property as a natural remedy for the combinatorial complexity. Furthermore, we propose a visual–semantic hierarchical supervision scheme that explicitly aligns visual features with textual semantics across multiple levels of granularity, thereby enhancing relational reasoning. Extensive experiments on widely used SGG benchmarks, evaluated in both closed- and open-vocabulary settings, demonstrate consistent improvements in accuracy and generalization over state-of-the-art methods.