Multimedia recommendation involves personalized ranking tasks, where multimedia content is usually represented using a generic encoder. However, these generic representations introduce spurious correlations that fail to reveal users' true preferences. Existing works attempt to alleviate this problem by learning invariant representations, but overlook the balance between independent and identically distributed (IID) and out-of-distribution (OOD) generalization. In this paper, we propose a framework called Pareto Invariant Representation Learning (PaInvRL) to mitigate the impact of spurious correlations from an IID-OOD multi-objective optimization perspective, by learning invariant representations (intrinsic factors that attract user attention) and variant representations (other factors) simultaneously. Specifically, PaInvRL includes three iteratively executed modules: (i) heterogeneous identification module, which identifies the heterogeneous environments to reflect distributional shifts for user-item interactions; (ii) invariant mask generation module, which learns invariant masks based on the Pareto-optimal solutions that minimize the adaptive weighted Invariant Risk Minimization (IRM) and Empirical Risk (ERM) losses; (iii) convert module, which generates both variant representations and item-invariant representations for training a multi-modal recommendation model that mitigates spurious correlations and balances the generalization performance within and cross the environmental distributions. We compare the proposed PaInvRL with state-of-the-art recommendation models on three public multimedia recommendation datasets (Movielens, Tiktok, and Kwai), and the experimental results validate the effectiveness of PaInvRL for both within-and cross-environmental learning.
1 January 2024 in Frontiers of Information Technology & Electronic Engineering
Artificial intelligence generated content (AIGC) has emerged as an indispensable tool for producing large-scale content in various forms, such as images, t
Multi-sensor image fusion is used to combine the complementary information of source images from the multiple sensors. Recently, conventional image fusion schemes based on signal processing techniques have been studied extensively, and machine learning-based techniques have been introduced into image fusion because of the prominent advantages. In this work, a new multi-sensor image fusion method based on the support vector machine and principal component analysis is proposed. First, the key features of the source images are extracted by combining the sliding window technique and five effective evaluation indicators. Second, a trained support vector machine model is used to extract the focus region and the non-focus region of the source images according to the extracted image features, the fusion decision is therefore obtained for each source image. Then, the consistency verification operation is used to absorb a single singular point in the decisions of the trained classifier. Finally, a novel method based on principal component analysis and the multi-scale sliding window is proposed to handle the disputed areas in the fusion decision pair. Experiments are performed to verify the performance of the new combined method.
Controllable Image Synthesis (CIS) is a methodology that allows users to generate desired images or manipulate specific attributes of images by providing precise input conditions or modifying latent representations. In recent years, CIS has attracted considerable attention in the field of image processing, with significant advances in consistency, controllability and harmony. However, several challenges still remain, particularly regarding the fine-grained controllability and interpretability of synthesized images. In this paper, we comprehensively and systematically review the CIS from problem definition, taxonomy and evaluation systems to existing challenges and future research directions. First, the definition of CIS is given, and several representative deep generative models are introduced in detail. Second, the existing CIS methods are divided into three categories according to the different control manners used and discuss the typical work in each category critically. Furthermore, we introduce the public datasets and evaluation metrics commonly used in image synthesis and analyze the representative CIS methods. Finally, we present several open issues and discuss the future research direction of CIS.
20 December 2025 in Expert Systems with Applications