Reinforcement learning (RL) has shown great promise in optimizing long-term user interest in recommender systems. However, existing RL-based recommendation methods need a large number of interactions for each user to learn the recommendation policy. The challenge becomes more critical when recommending to new users who have a limited number of interactions. To that end, in this article, we address the cold-start challenge in the RL-based recommender systems by proposing a novel context-aware offline meta-level model-based RL approach for user adaptation. Our proposed approach learns to infer each user's preference with a user context variable that enables recommendation systems to better adapt to new users with limited contextual information. To improve adaptation efficiency, our approach learns to recover the user choice function and reward from limited contextual information through an inverse RL method, which is used to assist the training of a meta-level recommendation agent. To avoid the need for online interaction, the proposed method is trained using historically collected offline data. Moreover, to tackle the challenge of offline policy training, we introduce a mutual information constraint between the user model and recommendation agent. Evaluation results show the superiority of our developed offline policy learning method when adapting to new users with limited contextual information. In addition, we provide a theoretical analysis of the recommendation performance bound.
Given the remarkable success of personalized recommendations on digital platforms, brick-and-mortar businesses are increasingly exploring artificial intelligence (AI)-powered recommendation services in physical spaces. To address this emerging need, our study introduces a generalized recommendation problem, termed point-of-interest (POI) recommendations in physical spaces with pedestrian movement (P3M). Applicable scenarios for P3M include store recommendations in shopping malls, product shelf recommendations in hypermarkets, and so on. A critical impediment in P3M is exposure bias: When the exposure likelihood of items to users is unevenly distributed, indiscriminately treating all unobserved interactions as negative feedback introduces bias to the learning of recommender systems. To address this issue, we propose a novel recommendation method, unbiased movement-aware pairwise ranking (UMPR), which integrates pedestrian movement modeling with unbiased pairwise learning to achieve effective and unbiased recommendations. Using real-world shopping mall data, we demonstrate that UMPR not only delivers more accurate recommendations compared to state-of-the-art methods but also brings added monetary value for mall owners and promotes humanistic fairness across store tenants. Overall, our study emphasizes the importance of mitigating exposure bias through pedestrian movement modeling, advancing the field of recommendations in physical spaces.