6 February 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Egocentric early action prediction aims to recognize actions from the first-person view by only observing a partial video segment, which is challenging due to the limited context information of the partial video. In this article, to tackle the egocentric early action prediction problem, we propose a novel multi-modal adversarial knowledge distillation framework. In particular, our approach involves a teacher network to learn the enhanced representation of the partial video by considering the future unobserved video segment, and a student network to mimic the teacher network to produce the powerful representation of the partial video and based on that predicting the action label. To promote the knowledge distillation between the teacher and the student network, we seamlessly integrate adversarial learning with latent and discriminative knowledge regularizations encouraging the learned representations of the partial video to be more informative and discriminative toward the action prediction. Finally, we devise a multi-modal fusion module toward comprehensively predicting the action label. Extensive experiments on two public egocentric datasets validate the superiority of our method over the state-of-the-art methods. We have released the codes and involved parameters to benefit other researchers. 1
7 February 2023 in ACM Transactions on Information Systems
Recommender systems can automatically recommend users with items that they probably like. The goal of them is to model the user-item interaction by effectively representing the users and items. Existing methods have primarily learned the user’s preferences and item’s features with vectorized embeddings, and modeled the user’s general preferences to items by the interaction of them. In fact, users have their specific preferences to item attributes and different preferences are usually related. Therefore, exploring the fine-grained preferences as well as modeling the relationships among user’s different preferences could improve the recommendation performance. Toward this end, we propose a dual preference distribution learning framework (DUPLE) , which aims to jointly learn a general preference distribution and a specific preference distribution for a given user, where the former corresponds to the user’s general preference to items and the latter refers to the user’s specific preference to item attributes. Notably, the mean vector of each Gaussian distribution can capture the user’s preferences, and the covariance matrix can learn their relationship. Moreover, we can summarize a preferred attribute profile for each user, depicting his/her preferred item attributes. We then can provide the explanation for each recommended item by checking the overlap between its attributes and the user’s preferred attribute profile. Extensive quantitative and qualitative experiments on six public datasets demonstrate the effectiveness and explainability of the DUPLE method.
4 July 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Few-shot food recognition aims to first train a meta-model based on an extensive labeled dataset, and then adapt it to recognize novel food classes with limited labeled data. Although existing studies have achieved compelling success, they still heavily relied on a large number of labeled food data for training the initial meta-model. To save the annotation cost, we propose the unsupervised food recognition task, which aims to train a meta-model using only unlabeled food data. Due to the two challenges presented in food images: 1) high intra-class variations and 2) high inter-class similarity, directly applying existing unsupervised few-shot learning methods could result in sub-optimal results. Towards this end, we propose a novel framework, i.e., Unsupervsied Few-shot Food Recognition with Intra-class Variation and Inter-class Similarity (UFFR-IVIS). It consists of two key components: (1) dual diversity-injected support/query representation learning that introduces instance-level and representation-level diversities for the representation learning of support/query instance to model the characteristics of high intra-class variation; and (2) dual regularization-enhanced meta learning that designs two regularizations: auxiliary task-based intra-class regularization and similarity-guided inter-class regularization to regularize the intra-class variation and inter-class similarity modeling, respectively. Extensive experiments on two food datasets demonstrate the superiority of our UFFR-IVIS.
1 January 2026 in IEEE Transactions on Circuits and Systems for Video Technology