Xiaolu Zhang

Xiaolu Zhang

Auto-generated profile, based on publicly available information

Xiaolu Zhang

Ant Group CO Ltd, Hangzhou, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 May 2024
Verified
Conference Paper
Open access
JHJun Hu
Jun Hu
WXWenwen Xia
Wenwen Xia
XZXiaolu Zhang
Xiaolu Zhang
CFChilin Fu
Chilin Fu
WWWeichang Wu
Weichang Wu
JZJun Zhou
Jun Zhou
Sequential recommendation systems (SRS) are crucial in various applications as they enable users to discover relevant items based on their past interactions. Recent advancements involving large language models (LLMs) have shown significant promise in addressing intricate recommendation challenges. However, these efforts exhibit certain limitations. Specifically, directly extracting representations from an LLM based on items' textual features and feeding them into a sequential model hold no guarantee that the semantic information of texts could be preserved in these representations. Additionally, concatenating textual descriptions of all items in an item sequence into a long text and feeding it into an LLM for recommendation results in lengthy token sequences, which largely diminishes the practical efficiency. In this paper, we introduce SAID, a framework that utilizes LLMs to explicitly learn Semantically Aligned item ID embeddings based on texts. For each item, SAID employs a projector module to transform an item ID into an embedding vector, which will be fed into an LLM to elicit the exact descriptive text tokens accompanied by the item. The item embeddings are forced to preserve fine-grained semantic information of textual descriptions. Further, the learned embeddings can be integrated with lightweight downstream sequential models for practical recommendations. In this way, SAID circumvents lengthy token sequences in previous works, reducing resources required in industrial scenarios and also achieving superior recommendation performance. Experiments on six public datasets demonstrate that SAID outperforms baselines by about 5% to 15% in terms of NDCG@10. Moreover, SAID has been deployed in Alipay's online advertising platform, achieving a 3.07% relative improvement of cost per mille (CPM) over baselines, with an online response time of under 20 milliseconds.

10 July 2025 in ACM Transactions on Information Systems
Verified
Journal Article
Open access
ZTZuoli Tang
Zuoli Tang
ZHZhaoxin Huan
Zhaoxin Huan
ZLZihao Li
Zihao Li
XZXiaolu Zhang
Xiaolu Zhang
JHJun Hu
Jun Hu
CLChenliang Li
Chenliang Li
Sequential recommendation systems aim to predict users’ next likely interaction based on their history. However, these systems face data sparsity and cold-start problems. Utilizing data from other domains, known as multi-domain methods, is useful for alleviating these problems. However, traditional multi-domain methods rely on meaningless ID-based item representation, which makes it difficult to align items with similar meanings from different domains, yielding sup-optimal knowledge transfer. This article introduces LLM-Rec , a framework that utilizes pre-trained Large Language Models (LLMs) for domain-agnostic recommendation. Specifically, we mix user’s behaviors from multiple domains and concatenate item titles into a sentence, then use LLMs for generating user and item representations. By mixing behaviors across different domains, we can exploit the knowledge encoded in LLMs to bridge the semantic across over multi-domain behaviors, thus obtaining semantically rich representations and improving performance in all domains. Furthermore, we explore the underlying reasons why LLMs are effective and investigate whether LLMs can understand the semantic correlations as the recommendation model, and if advanced techniques like scaling laws in NLP also work in recommendations. We conduct extensive experiments with LLMs ranging from 40 M to 6.7 B to answer the above questions and to verify the effectiveness of LLM-Rec in multi-domain recommendation. The source code is available at https://github.com/WHUIR/LLMRec .

10 July 2024
Verified
Conference Paper
Open access
BGBinzong Geng
Binzong Geng
ZHZhaoxin Huan
Zhaoxin Huan
XZXiaolu Zhang
Xiaolu Zhang
YHYong He
Yong He
LZLiang Zhang
Liang Zhang
LMLinjian Mo
Linjian Mo
With the rise of large language models (LLMs), recent works have leveraged LLMs to improve the performance of click-through rate (CTR) prediction. However, we argue that a critical obstacle remains in deploying LLMs for practical use: the efficiency of LLMs when processing long textual user behaviors. As user sequences grow longer, the current efficiency of LLMs is inadequate for training on billions of users and items. To break through the efficiency barrier of LLMs, we propose Behavior Aggregated Hierarchical Encoding (BAHE) to enhance the efficiency of LLM-based CTR modeling. Specifically, BAHE proposes a novel hierarchical architecture that decouples the encoding of user behaviors from inter-behavior interactions. Firstly, to prevent computational redundancy from repeated encoding of identical user behaviors, BAHE employs the LLM's pre-trained shallow layers to extract embeddings of the most granular, atomic user behaviors from extensive user sequences and stores them in the offline database. Subsequently, the deeper, trainable layers of the LLM facilitate intricate inter-behavior interactions, thereby generating comprehensive user embeddings. This separation allows the learning of high-level user representations to be independent of low-level behavior encoding, significantly reducing computational complexity. Finally, these refined user embeddings, in conjunction with correspondingly processed item embeddings, are incorporated into the CTR model to compute the CTR scores. Extensive experimental results show that BAHE reduces training time and memory by five times for CTR models using LLMs, especially with longer user sequences. BAHE has been deployed in a real-world system, allowing for daily updates of 50 million CTR data on 8 A100 GPUs, making LLMs practical for industrial CTR prediction.

13 May 2024
Verified
Conference Paper
Open access
SWShuhan Wang
Shuhan Wang
BSBin Shen
Bin Shen
XMXu Min
Xu Min
YHYong He
Yong He
XZXiaolu Zhang
Xiaolu Zhang
LMLinjian Mo
Linjian Mo
Combining contextual information (i.e., side information) of items beyond IDs has become an important way to improve the performance in recommender systems. Existing self-attention-based side information fusion methods can be categorized into early, late, and hybrid fusion. In practice, naive early fusion may interfere with the representation of IDs, resulting in negative effects, while late fusion misses effective interactions between IDs and side information. Some hybrid methods have been proposed to address these issues, but they only utilize side information in calculating attention scores, which may lead to information loss. To harness the full potential of side information without noisy interference, we propose an Aligned Side Information Fusion (ASIF) method for sequential recommendation, consisting of two parts: Fused Attention with Untied Positions and Representation Alignment. Specifically, we first decouple the positions to exclude the noisy interference in the attention scores. Secondly, we adopt the contrastive objective to maintain the semantic consistency between IDs and side information and then employ orthogonal decomposition to extract the homogeneous parts. By aligning the representations and fusing them together, ASIF makes full use of the side information without interfering with IDs. Offline experimental results on four datasets demonstrate the superiority of ASIF. Additionally, we successfully deployed the model in Alipay's advertising system and achieved 1.09% and 1.86% improvements on clicks and Cost Per Mille (CPM).

18 July 2023
Verified
Conference Paper
Open access
ZTZuoli Tang
Zuoli Tang
LWLin Wang
Lin Wang
LZLixin Zou
Lixin Zou
XZXiaolu Zhang
Xiaolu Zhang
JZJun Zhou
Jun Zhou
CLChenliang Li
Chenliang Li
The pre-training paradigm, i.e., learning universal knowledge across a wide spectrum of domains, has increasingly become a new de-facto practice in many fields, especially for transferring to new domains. The recent progress includes universal pre-training solutions for recommendation. However, we argue that the common treatment utilizing the masked language modeling or simple data augmentation via contrastive learning is not sufficient for pre-training a recommender system, since a user's intent could be more complex than predicting the next word or item. It is more intuitive to go a step further by devising the multi-interest driven pre-training framework for universal user understanding. Nevertheless, incorporating multi-interest modeling in recommender system pre-training is non-trivial due to the dynamic, contextual, and temporary nature of the user interests, particularly when the users are from different domains. The limited effort on this line has greatly rendered it as an open question. In this paper, we propose a novel Multi-Interest Pre-training with Sparse Capsule framework (named Miracle). Miracle performs a universal multi-interest modeling with a sparse capsule network and an interest-aware pre-training task. Specifically, we utilize a text-aware item embedding module, including an MoE adaptor and a deeply-contextual encoding component, to model contextual and transferable item representations. Then, we propose a sparse interest activation mechanism coupled with a position-aware capsule network for adaptive interest extraction. Furthermore, an interest-level contrastive pre-training task is introduced to guide the sparse capsule network to learn universal interests precisely. We conduct extensive experiments on eleven real-world datasets and eight baselines. The results show that our method significantly outperforms a series of SOTA on these benchmark datasets. The code is available at https://github.com/WHUIR/Miracle.

21 October 2023
Verified
Conference Paper
Open access
YXYoushao Xiao
Youshao Xiao
SZShangchun Zhao
Shangchun Zhao
ZZZhenglei Zhou
Zhenglei Zhou
ZHZhaoxin Huan
Zhaoxin Huan
LJLin Ju
Lin Ju
JZJun Zhou
Jun Zhou
Recently, a new paradigm, meta learning, has been widely applied to Deep Learning Recommendation Models (DLRM) and significantly improves statistical performance, especially in cold-start scenarios. However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster. It is because the conventional deep learning pipeline is not optimized for two task-specific datasets and two update loops in meta learning. This paper provides a high-performance framework for large-scale training for Optimization-based Meta DLRM models over the G PU cluster, namely G -Meta. Firstly, G-Meta utilizes both data parallelism and model parallelism with careful orchestration regarding computation and communication efficiency, to enable high-speed distributed training. Secondly, it proposes a Meta-IO pipeline for efficient data ingestion to alleviate the I/O bottleneck. Various experimental results show that G-Meta achieves notable training speed without loss of statistical performance. Since early 2022, G-Meta has been deployed in Alipay's core advertising and recommender system, shrinking the continuous delivery of models by four times. It also obtains 6.48% improvement in Conversion Rate (CVR) and 1.06% increase in CPM (Cost Per Mille) in Alipay's homepage display advertising, with the benefit of larger training samples and tasks.