Houari Sahraoui

Houari Sahraoui

Auto-generated profile, based on publicly available information

Houari Sahraoui

DIRO, Université de Montréal, Montréal, Canada

Research Focus & Metrics

Academic Career Length

28Years (1999-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 August 2025 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
MWMartin Weyssow
Martin Weyssow
XZXin Zhou
Xin Zhou
KKKisub Kim
Kisub Kim
DLDavid Lo
David Lo
HSHouari Sahraoui
Houari Sahraoui
Large language models (LLMs) demonstrate impressive capabilities to generate accurate code snippets given natural language intents in a zero-shot manner, i.e., without the need for specific fine-tuning. While prior studies have highlighted the advantages of fine-tuning LLMs, this process incurs high computational costs, making it impractical in resource-scarce environments, particularly for models with billions of parameters. To address these challenges, previous research explored in-context learning (ICL) and retrieval-augmented generation (RAG) as strategies to guide the LLM generative process with task-specific prompt examples. However, ICL and RAG introduce inconveniences, such as the need for designing contextually relevant prompts and the absence of learning task-specific parameters, thereby limiting downstream task performance. In this context, we foresee parameter-efficient fine-tuning (PEFT) as a promising approach to efficiently specialize LLMs to task-specific data while maintaining reasonable resource consumption. In this article, we deliver a comprehensive study of PEFT techniques for LLMs in the context of automated code generation. Our comprehensive investigation of PEFT techniques for LLMs reveals their superiority and potential over ICL and RAG across a diverse set of LLMs and three representative Python code generation datasets: Conala, CodeAlpacaPy, and APPS. Furthermore, our study highlights the potential for tuning larger LLMs and significant reductions in memory usage by combining PEFT with quantization. Therefore, this study opens opportunities for broader applications of PEFT in software engineering scenarios.

12 February 2022 in Software and Systems Modeling
Verified
Journal Article
Closed access
PDF file
MWMartin Weyssow
Martin Weyssow
HSHouari Sahraoui
Houari Sahraoui
ESEugene Syriani
Eugene Syriani

13 February 2026 in ACM Transactions on Software Engineering and Methodology
Verified
Journal Article
Open access
MWMartin Weyssow
Martin Weyssow
AKAton Kamanda
Aton Kamanda
XZXin Zhou
Xin Zhou
HSHouari Sahraoui
Houari Sahraoui
Evaluating the alignment of large language models (LLMs) with user-defined coding preferences is a challenging endeavor that requires a deep assessment of LLMs’ outputs. Existing methods and benchmarks rely primarily on automated metrics and static analysis tools, which often fail to capture the nuances of user instructions and LLM outputs. To address this gap, we introduce the LLM-as-a-Judge evaluation framework and present CodeUltraFeedback, a comprehensive dataset for assessing and improving LLM alignment with coding preferences. CodeUltraFeedback consists of 10,000 coding instructions, each annotated with four responses generated from a diverse pool of 14 LLMs. These responses are annotated using GPT-3.5 as a judge, with both ranking-based scores and detailed textual feedback across five distinct coding preferences. Our analysis reveals that responses from GPT-3.5 and GPT-4 are consistently rated higher than those from open-weight models, underscoring substantial alignment gaps between closed- and open-weight LLMs. In turn, we explore the usage of CodeUltraFeedback as feedback data to fine-tune and align CodeLlama-7B-Instruct using supervised fine-tuning (SFT) and reinforcement learning from AI feedback (RLAIF) with direct preference optimization (DPO). The resulting aligned model achieves an average alignment improvement of 22.7% and 29.7% when evaluated with GPT-3.5 and GPT-4 judges, respectively. Notably, our aligned CodeLlama-7B-Instruct surpasses much larger models, such as CodeLlama-13B and 34B, in alignment with coding preferences. Despite not being explicitly trained for functional correctness, it also achieves a 10.5% and 26.6% relative improvement in Pass@1 and Pass@10 on the HumanEval+ benchmark. Our contributions demonstrate the practical value of preference tuning in code generation and set the stage for further progress in model alignment and RLAIF for automated software engineering.

22 September 2010 in Software and Systems Modeling
Verified
Journal Article
Closed access
PDF file
MKMarouane Kessentini
Marouane Kessentini
HSHouari Sahraoui
Houari Sahraoui
Mounir BoukadoumMounir Boukadoum
Mounir Boukadoum
OOOmar Ben Omar
Omar Ben Omar

1 July 2015 in Journal of Systems and Software
Verified
Journal Article
Closed access
AOAli Ouni
Ali Ouni
MKMarouane Kessentini
Marouane Kessentini
HSHouari Sahraoui
Houari Sahraoui
KIKatsuro Inoue
Katsuro Inoue
MHMohamed Salah Hamdi
Mohamed Salah Hamdi

10 October 2022
Verified
Conference Paper
Open access
JLJosé Antonio Hernández López
José Antonio Hernández López
MWMartin Weyssow
Martin Weyssow
JCJesús Sánchez Cuadrado
Jesús Sánchez Cuadrado
HSHouari Sahraoui
Houari Sahraoui
The objective of pre-trained language models is to learn contextual representations of textual data. Pre-trained language models have become mainstream in natural language processing and code modeling. Using probes, a technique to study the linguistic properties of hidden vector spaces, previous works have shown that these pre-trained language models encode simple linguistic properties in their hidden representations. However, none of the previous work assessed whether these models encode the whole grammatical structure of a programming language. In this paper, we prove the existence of a syntactic subspace, lying in the hidden representations of pre-trained language models, which contain the syntactic information of the programming language. We show that this subspace can be extracted from the models’ representations and define a novel probing method, the AST-Probe, that enables recovering the whole abstract syntax tree (AST) of an input code snippet. In our experimentations, we show that this syntactic subspace exists in five state-of-the-art pre-trained language models. In addition, we highlight that the middle layers of the models are the ones that encode most of the AST information. Finally, we estimate the optimal size of this syntactic subspace and show that its dimension is substantially lower than those of the models’ representation spaces. This suggests that pre-trained language models use a small part of their representation spaces to encode syntactic information of the programming languages.