27 October 2020 in IEEE Transactions on Neural Networks and Learning Systems
Negative sampling plays an important role in ranking-based recommender models. However, most existing sampling methods cannot generate informative item pairs with positive and negative instances due to two limitations: 1) they merely treat observed items as positive instances, ignoring the existence of potential positive items (i.e., nonobserved items users may prefer) and the probability of observed but noisy items and 2) they fail to capture the relationship between positive and negative items during negative sampling, which may cause the unexpected selection of potential positive items. In this article, we introduce a dynamic sampling strategy to search informative item pairs. Specifically, we first sample a positive instance from all the items by leveraging the overall features of user's observed items. Then, we strategically select a negative instance by considering its correlation with the sampled positive one. Formally, we propose an item pair generative adversarial network named IPGAN, where our sampling strategy is realized in two generative models for positive and negative instances, respectively. In addition, IPGAN can also ensure that the sampled item pairs are informative relative to the ground truth by a discriminative model. What is more, we propose a batch-training approach to further enhance both user and item modeling by alleviating the special bias (noise) from different users. This approach can also significantly accelerate the process of model training compared with classical GAN method for recommendation. Experimental results on three real data sets show that our approach outperforms other state-of-the-art approaches in terms of recommendation accuracy.
9 June 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Two-Tower Vision–Language Models (VLMs) have demonstrated strong performance across various downstream VL tasks. While BridgeTower further enhances performance by building bridges between encoders, it (i) suffers from ineffective layer-by-layer utilization of unimodal representations, (ii) restricts the flexible exploitation of different levels of unimodal semantic knowledge, and (iii) is limited to the evaluation on traditional low-resolution datasets only with the Two-Tower VLM architecture. In this work, we propose Manager, a lightweight, efficient and effective plugin that adaptively aggregates insights from different levels of pre-trained unimodal experts to facilitate more comprehensive VL alignment and fusion. First, under the Two-Tower VLM architecture, we introduce ManagerTower, a novel VLM that introduces the manager in each cross-modal layer. Whether with or without VL pre-training, ManagerTower outperforms previous strong baselines and achieves superior performance on 4 downstream VL tasks. Moreover, we extend our exploration to the latest Multimodal Large Language Model (MLLM) architecture.We demonstrate that LLaVA-OV-Manager significantly boosts the zero-shot performance of LLaVA-OV across different categories of capabilities, images, and resolutions on 20 downstream datasets, whether the multi-grid algorithm is enabled or not. In-depth analysis reveals that both our manager and the multi-grid algorithm can be viewed as a plugin that improves the visual representation by capturing more diverse visual details from two orthogonal perspectives (depth and width). Their synergy can mitigate the semantic ambiguity caused by the multi-grid algorithm and further improve performance. Code and models are available at https://github.com/LooperXX/ManagerTower.