Thibault Formal

Thibault Formal

Auto-generated profile, based on publicly available information

Thibault Formal

Naver Labs Europe, Meylan, France

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 April 2024 in ACM Transactions on Information Systems
Verified
Journal Article
Open access
TFThibault Formal
Thibault Formal
CLCarlos Lassance
Carlos Lassance
BPBenjamin Piwowarski
Benjamin Piwowarski
SCStéphane Clinchant
Stéphane Clinchant
Sparse representation learning based on Pre-trained Language Models has seen a growing interest in Information Retrieval. Such approaches can take advantage of the proven efficiency of inverted indexes and inherit desirable IR priors such as explicit lexical matching or some degree of interpretability. In this work, we thoroughly develop the framework of sparse representation learning in IR, which unifies term weighting and expansion in a supervised setting. We then build on SPLADE—a sparse expansion-based retriever—and show to which extent it is able to benefit from the same training improvements as dense bi-encoders by studying the effect of distillation, hard negative mining, as well as the Pre-trained Language Model’s initialization on itseffectiveness, leading to state-of-the-art results in both in- and out-of-domain evaluation settings (SPLADE++). We furthermore proposeefficiencyimprovements, allowing us to reach latency requirements on par with traditional keyword-based approaches (Efficient-SPLADE).

9 August 2023
Verified
Conference Paper
Open access
GFGuglielmo Faggioli
Guglielmo Faggioli
TFThibault Formal
Thibault Formal
SLSimon Lupart
Simon Lupart
SMStefano Marchesin
Stefano Marchesin
SCStephane Clinchant
Stephane Clinchant
BPBenjamin Piwowarski
Benjamin Piwowarski
In this work, we propose a novel framework to devise features that can be used by Query Performance Prediction (QPP) models for Neural Information Retrieval (NIR). Using the proposed framework as a periodic table of QPP components, practitioners can devise new predictors better suited for NIR. Through the framework, we detail what challenges and opportunities arise for QPPs at different stages of the NIR pipeline. We show the potential of the proposed framework by using it to devise two types of novel predictors. The first one, named MEMory-based QPP (MEM-QPP), exploits the similarity between test and train queries to measure how much a NIR system can memorize. The second adapts traditional QPPs into NIR-oriented ones by computing the query-corpus semantic similarity. By exploiting the inherent nature of NIR systems, the proposed predictors overcome, under various setups, the current State of the Art, highlighting -- at the same time -- the versatility of the framework in describing different types of QPPs.

10 July 2024
Verified
Conference Paper
Open access
TFThibault Formal
Thibault Formal
SCStéphane Clinchant
Stéphane Clinchant
HDHervé Déjean
Hervé Déjean
CLCarlos Lassance
Carlos Lassance
The late interaction paradigm introduced with ColBERT stands out in the neural Information Retrieval space, offering a compelling effectiveness-efficiency trade-off across many benchmarks. Efficient late interaction retrieval is based on an optimized multi-step strategy, where an approximate search first identifies a set of candidate documents to re-rank exactly. In this work, we introduce SPLATE, a simple and lightweight adaptation of the ColBERTv2 model which learns an "MLM adapter'', mapping its frozen token embeddings to a sparse vocabulary space with a partially learned SPLADE module. This allows us to perform the candidate generation step in late interaction pipelines with traditional sparse retrieval techniques, making it particularly appealing for running ColBERT in CPU environments. Our SPLATE ColBERTv2 pipeline achieves the same effectiveness as the PLAID ColBERTv2 engine by re-ranking 50 documents that can be retrieved under 10ms.

18 July 2023
Verified
Conference Paper
Open access
HDHervé Déjean
Hervé Déjean
SCStephane Clinchant
Stephane Clinchant
CLCarlos Lassance
Carlos Lassance
SLSimon Lupart
Simon Lupart
TFThibault Formal
Thibault Formal
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval. Recent models such as CoCondenser, RetroMAE, and LexMAE argue that the MLM task is not sufficient enough to pre-train a transformer network for retrieval and hence propose various tasks to do so. Intrigued by those novel methods, we noticed that all these models used different finetuning protocols, making it hard to assess the benefits of middle training. We propose in this paper a benchmark of CoCondenser, RetroMAE, and LexMAE, under the same finetuning conditions. We compare both dense and sparse approaches under various finetuning protocols and middle training on different collections (MS MARCO, Wikipedia). We use additional middle training baselines, such as a standard MLM finetuning on the retrieval collection, optionally augmented by a CLS predicting the passage term frequency. For the sparse approach, our study reveals that there is almost no statistical difference between those methods: the more effective the finetuning procedure is, the less difference there is between those models. For the dense approach, RetroMAE using MS MARCO as middle-training collection shows excellent results in almost all the settings. Finally, we show that middle training on the retrieval collection, thus adapting the language model to it, is a critical factor. Overall, a better experimental setup should be adopted to evaluate middle training methods.