As one of the most advanced techniques in AI, Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge, providing huge convenience for numerous tasks. Particularly in the era of AI-Generated Content (AIGC), the powerful capacity of retrieval in providing additional knowledge enables RAG to assist existing generative AI in producing high-quality outputs. Recently, Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation, while still facing inherent limitations such as hallucinations and out-of-date internal knowledge. Given the powerful abilities of RAG in providing the latest and helpful auxiliary information, Retrieval-Augmented Large Language Models (RA-LLMs) have emerged to harness external and authoritative knowledge bases, rather than solely relying on the model's internal knowledge, to augment the quality of the generated content of LLMs. In this survey, we comprehensively review existing research studies in RA-LLMs, covering three primary technical perspectives: Furthermore, to deliver deeper insights, we discuss current limitations and several promising directions for future research. Updated information about this survey can be found at: https://advanced-recommender-systems.github.io/RAG-Meets-LLMs/

With the prosperity of e-commerce and web applications, Recommender Systems (RecSys) have become an indispensable and important component, providing personalized suggestions that cater to user preferences. While Deep Neural Networks (DNNs) have achieved significant advancements in enhancing recommender systems, these DNN-based methods still exhibit some limitations, such as inferior capabilities to effectively capture textual side information about users and items, difficulties in generalization to various recommendation scenarios, and reasoning on their predictions, etc. Meanwhile, the development of Large Language Models (LLMs), such as ChatGPT and GPT-4, has revolutionized the fields of Natural Language Processing (NLP) and Artificial Intelligence (AI), due to their remarkable abilities in fundamental responsibilities of language understanding and generation, as well as impressive generalization capabilities and reasoning skills. As a result, recent studies have actively attempted to harness the power of LLMs to enhance recommender systems. Given the rapid evolution of this research direction in recommender systems, there is a pressing need for a systematic overview that summarizes existing LLM-empowered recommender systems. Therefore, in this survey, we comprehensively review LLM-empowered recommender systems from various perspectives including pre-training, fine-tuning, and prompting paradigms. More specifically, we first introduce the representative methods to learn user and item representations, leveraging LLMs as feature encoders. Then, we systematically review the emerging advanced techniques of LLMs for enhancing recommender systems from three paradigms, namely pre-training, fine-tuning, and prompting. Finally, we comprehensively discuss the promising future directions in this emerging field.

Unsupervised learning with generative adversarial networks (GANs) has proven hugely successful. Regular GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function. However, we found that this loss function may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose in this paper the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson $x^{2}$ divergence. There are two benefits of LSGANs over regular GANs. First, LSGANs are able to generate higher quality images than regular GANs. Second, LSGANs perform more stable during the learning process. We train LSGANs on several datasets, and the experimental results show that the images generated by LSGANs are of better quality than regular GANs. Furthermore, we evaluate the stability of LSGANs in two groups. One is to compare between LSGANs and regular GANs without gradient penalty. The other one is to compare between LSGANs with gradient penalty and WGANs with gradient penalty. We conduct four experiments to illustrate the stability of LSGANs. The other one is to compare between LSGANs with gradient penalty (LSGANs-GP) and WGANs with gradient penalty (WGANs-GP). The experimental results show that LSGANs-GP succeed in training for all the difficult architectures used in WGANs-GP, including 101-layer ResNet.

Nowadays, most data about social networks, users shopping behaviors, and inter-item relationships etc., can be represented as graph structure. Graph Neural Networks (GNNs) have shown great success in learning meaningful representations for graph data by naturally integrating node information and topological structures. Data used in making social recommendations can also be represented as graph data in the form of user-user social graphs and user-item graphs. In addition, the relationships between items can be represented as graph data, denoted as item-item graph. GNNs provide an unprecedented opportunity to advance social recommendations, yet there are considerable challenges in building GNNs-based social recommendations based on this modelling framework in that (1) users (items) are simultaneously involved in the user-item graph and user-user social graph (item-item graph); (2) user-item graphs not only contain user-item interactions but also include users' opinions on items; and (3) the nature of social relations are heterogeneous among users. In this paper, we propose a novel graph neural network framework (GraphRec+) for social recommendations, which is able to coherently model graph data in order to learn better user and item representations. Comprehensive experiments on three real-world datasets show the effectiveness of the proposed framework.

Molecule discovery plays a crucial role in various scientific fields, advancing the design of tailored materials and drugs, which contributes to the development of society and human well-being. Specifically, molecule-caption translation is an important task for molecule discovery, aligning human understanding with molecular space. However, most of the existing methods heavily rely on domain experts, require excessive computational cost, or suffer from sub-optimal performance. On the other hand, Large Language Models (LLMs), like ChatGPT, have shown remarkable performance in various cross-modal tasks due to their powerful capabilities in natural language understanding, generalization, and in-context learning (ICL), which provides unprecedented opportunities to advance molecule discovery. Despite several previous works trying to apply LLMs in this task, the lack of domain-specific corpus and difficulties in training specialized LLMs still remain challenges. In this work, we propose a novel LLM-based framework ( MolReGPT ) for molecule-caption translation, where an In-Context Few-Shot Molecule Learning paradigm is introduced to empower molecule discovery with LLMs like ChatGPT to perform their in-context learning capability without domain-specific pre-training and fine-tuning. MolReGPT leverages the principle of molecular similarity to retrieve similar molecules and their text descriptions from a local database to enable LLMs to learn the task knowledge from context examples. We evaluate the effectiveness of MolReGPT on molecule-caption translation, including molecule understanding and text-based molecule generation. Experimental results show that compared to fine-tuned models, MolReGPT outperforms MolT5-base and is comparable to MolT5-large without additional training. To the best of our knowledge, MolReGPT is the first work to leverage LLMs via in-context learning in molecule-caption translation for advancing molecule discovery. Our work expands the scope of LLM applications, as well as providing a new paradigm for molecule discovery and design.

We study the joint inventory replenishment and pricing problem for production systems with random demand and yield. More specifically, we analyze the following single-item, periodic-review model. Demands in consecutive periods are independent random variables and their distributions are price sensitive. The production yield is uncertain so that the quantity received from a replenishment is a random variable whose distribution depends on the production quantity. Stockouts are fully backlogged. Our problem is to characterize the optimal dynamic policy that simultaneously determines the production quantity and the price for each period to maximize the total discounted profit. We show that the optimal replenishment policy is of a threshold type, i.e., it is optimal to produce if and only if the starting inventory in a period is below a threshold value, and that both the optimal production quantity and the optimal price in each period are decreasing in the starting inventory. We further study the operational effects of uncertain yield. We prove that, in the single-period case, the threshold of replenishment is independent of the yield variability, and, in the multiperiod case, it is higher in a system with uncertain yield than in one with certain yield. In addition, the system with uncertain yield always charges a higher price.