Shuai Wang

Shuai Wang

Auto-generated profile, based on publicly available information

Shuai Wang

The University of Queensland, Brisbane, Australia

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 July 2023
Verified
Conference Paper
Open access
SWShuai Wang
Shuai Wang
HSHarrisen Scells
Harrisen Scells
BKBevan Koopman
Bevan Koopman
GZGuido Zuccon
Guido Zuccon

20 March 2024
Verified
Book Chapter
Closed access
PDF file
SWShuai Wang
Shuai Wang
HSHarrisen Scells
Harrisen Scells
SZShengyao Zhuang
Shengyao Zhuang
MPMartin Potthast
Martin Potthast
BKBevan Koopman
Bevan Koopman
GZGuido Zuccon
Guido Zuccon

10 July 2024
Verified
Conference Paper
Open access
SWShuai Wang
Shuai Wang
EKEkaterina Khramtsova
Ekaterina Khramtsova
SZShengyao Zhuang
Shengyao Zhuang
GZGuido Zuccon
Guido Zuccon
Federated search systems aggregate results from multiple search engines, selecting appropriate sources to enhance result quality and align with user intent. With the increasing uptake of Retrieval-Augmented Generation (RAG) pipelines, federated search can play a pivotal role in sourcing relevant information across heterogeneous data sources to generate informed responses. However, existing datasets, such as those developed in the past TREC FedWeb tracks, predate the RAG paradigm shift and lack representation of modern information retrieval challenges. To bridge this gap, we present FeB4RAG, a novel dataset specifically designed for federated search within RAG frameworks. This dataset, derived from 16 sub-collections of the widely used BEIR benchmarking collection, includes 790 information requests (akin to conversational queries) tailored for chatbot applications, along with top results returned by each resource and associated LLM-derived relevance judgements. Additionally, to support the need for this collection, we demonstrate the impact on response generation of a high quality federated search system for RAG compared to a naive approach to federated search. We do so by comparing answers generated by the RAG pipeline with a qualitative side-by-side comparison. Our collection fosters and supports the development and evaluation of new federated search methods, especially in the context of RAG pipelines. The resource is publicly available at https://github.com/ielab/FeB4RAG.

10 July 2024
Verified
Conference Paper
Open access
LGLukas Gienapp
Lukas Gienapp
HSHarrisen Scells
Harrisen Scells
NDNiklas Deckers
Niklas Deckers
JBJanek Bevendorff
Janek Bevendorff
SWShuai Wang
Shuai Wang
MPMartin Potthast
Martin Potthast
Recent advances in large language models have enabled the development of viable generative retrieval systems. Instead of a traditional document ranking, generative retrieval systems often directly return a grounded generated text as a response to a query. Quantifying the utility of the textual responses is essential for appropriately evaluating such generative ad hoc retrieval. Yet, the established evaluation methodology for ranking-based ad hoc retrieval is not suited for the reliable and reproducible evaluation of generated responses. To lay a foundation for developing new evaluation methods for generative retrieval systems, we survey the relevant literature from the fields of information retrieval and natural language processing, identify search tasks and system architectures in generative retrieval, develop a new user model, and study its operationalization.

20 October 2022 in Intelligent Systems with Applications
Verified
Journal Article
Open access
SWShuai Wang
Shuai Wang
HSHarrisen Scells
Harrisen Scells
BKBevan Koopman
Bevan Koopman
GZGuido Zuccon
Guido Zuccon

15 December 2022
Verified
Conference Paper
Open access
SWShuai Wang
Shuai Wang
HSHarrisen Scells
Harrisen Scells
BKBevan Koopman
Bevan Koopman
GZGuido Zuccon
Guido Zuccon
Medical systematic reviews typically require assessing all the documents retrieved by a search. The reason is two-fold: the task aims for “total recall”; and documents retrieved using Boolean search are an unordered set, and thus it is unclear how an assessor could examine only a subset. Screening prioritisation is the process of ranking the (unordered) set of retrieved documents, allowing assessors to begin the downstream processes of the systematic review creation earlier, leading to earlier completion of the review, or even avoiding screening documents ranked least relevant. Screening prioritisation requires highly effective ranking methods. Pre-trained language models are state-of-the-art on many IR tasks but have yet to be applied to systematic review screening prioritisation. In this paper, we apply several pre-trained language models to the systematic review document ranking task, both directly and fine-tuned. An empirical analysis compares how effective neural methods compare to traditional methods for this task. We also investigate different types of document representations for neural methods and their impact on ranking performance. Our results show that BERT-based rankers outperform the current state-of-the-art screening prioritisation methods. However, BERT rankers and existing methods can actually be complementary, and thus, further improvements may be achieved if used in conjunction.