Retrieval-Augmented Generation (RAG) has recently emerged as a method to extend beyond the pre-trained knowledge of Large Language Models by augmenting the original prompt with relevant passages or documents retrieved by an Information Retrieval (IR) system. RAG has become increasingly important for Generative AI solutions, especially in enterprise settings or in any domain in which knowledge is constantly refreshed and cannot be memorized in the LLM. We argue here that the retrieval component of RAG systems, be it dense or sparse, deserves increased attention from the research community, and accordingly, we conduct the first comprehensive and systematic examination of the retrieval strategy of RAG systems. We focus, in particular, on the type of passages IR systems within a RAG solution should retrieve. Our analysis considers multiple factors, such as the relevance of the passages included in the prompt context, their position, and their number. One counter-intuitive finding of this work is that the retriever's highest-scoring documents that are not directly relevant to the query (e.g., do not contain the answer) negatively impact the effectiveness of the LLM. Even more surprising, we discovered that adding random documents in the prompt improves the LLM accuracy by up to 35%. These results highlight the need to investigate the appropriate strategies when integrating retrieval with LLMs, thereby laying the groundwork for future research in this area.

Forecasting geomagnetic indices represents a key point to develop warning systems for the mitigation of possible effects of severe geomagnetic storms on critical ground infrastructures. Here we focus on SYM‐H index, a proxy of the axially symmetric magnetic field disturbance at low and middle latitudes on the Earth's surface. To forecast SYM‐H, we built two artificial neural network (ANN) models and trained both of them on two different sets of input parameters including interplanetary magnetic field components and magnitude and differing for the presence or not of previous SYM‐H values. These ANN models differ in architecture being based on two conceptually different neural networks: the long short‐term memory (LSTM) and the convolutional neural network (CNN). Both networks are trained, validated, and tested on a total of 42 geomagnetic storms among the most intense that occurred between 1998 and 2018. Performance comparison of the two ANN models shows that (1) both are able to well forecast SYM‐H index 1 h in advance, with an accuracy of more than 95% in terms of the coefficient of determination R²; (2) the model based on LSTM is slightly more accurate than that based on CNN when including SYM‐H index at previous steps among the inputs; and (3) the model based on CNN has interesting potentialities being more accurate than that based on LSTM when not including SYM‐H index among the inputs. Predictions made including SYM‐H index among the inputs provide a root mean squared error on average 42% lower than that of predictions made without SYM‐H.

Sequential Recommender Systems (SRSs) have emerged as a highly efficient approach to recommendation systems. By leveraging sequential data, SRSs can identify temporal patterns in user behaviour, significantly improving recommendation accuracy and relevance. Ensuring the reproducibility of these models is paramount for advancing research and facilitating comparisons between them. Existing works exhibit shortcomings in reproducibility and replicability of results, leading to inconsistent statements across papers. Our work fills these gaps by standardising data pre-processing and model implementations, providing a comprehensive code resource, including a framework for developing SRSs and establishing a foundation for consistent and reproducible experimentation. We conduct extensive experiments on several benchmark datasets, comparing various SRSs implemented in our resource. We challenge prevailing performance benchmarks, offering new insights into the SR domain. For instance, SASRec does not consistently outperform GRU4Rec. On the contrary, when the number of model parameters becomes substantial, SASRec starts to clearly dominate all the other SRSs. This discrepancy underscores the significant impact that experimental configuration has on the outcomes and the importance of setting it up to ensure precise and comprehensive results. Failure to do so can lead to significantly flawed conclusions, highlighting the need for rigorous experimental design and analysis in SRS research. Our code is available at https://github.com/antoniopurificato/recsys_repro_conf.

Predicting the outcome of antiretroviral therapies (ART) for HIV-1 is a pressing clinical challenge, especially when the ART includes drugs with limited effectiveness data. This scarcity of data can arise either due to the introduction of a new drug to the market or due to limited use in clinical settings, resulting in clinical dataset with highly unbalanced therapy representation. To tackle this issue, we introduce a novel joint fusion model, which combines features from a Fully Connected (FC) Neural Network and a Graph Neural Network (GNN) in a multi-modality fashion. Our model uses both tabular data about genetic sequences and a knowledge base derived from Stanford drug-resistance mutation tables, which serve as benchmark references for deducing in-vivo treatment efficacy based on the viral genetic sequence. By leveraging this knowledge base structured as a graph, the GNN component enables our model to adapt to imbalanced data distributions and account for Out-of-Distribution (OoD) drugs. We evaluated these models' robustness against OoD drugs in the test set. Our comprehensive analysis demonstrates that the proposed model consistently outperforms the FC model. These results underscore the advantage of integrating Stanford scores in the model, thereby enhancing its generalizability and robustness, but also extending its utility in contributing in more informed clinical decisions with limited data availability. The source code is available at https://github.com/federicosiciliano/graph-ood-hiv.

In recent years, Retrieval Augmented Generation (RAG) systems have emerged as a pivotal component in the field of artificial intelligence, gaining significant attention and importance across various domains. These systems, which combine the strengths of information retrieval and generative models, have shown promise in enhancing the capabilities and performance of machine learning applications. However, despite their growing prominence, RAG systems are not without their limitations and continue to be in need of exploration and improvement. This workshop seeks to focus on the critical aspect of information retrieval and its integral role within RAG frameworks. We argue that current efforts have undervalued the role of Information Retrieval (IR) in the RAG and have concentrated their attention on the generative part. As the cornerstone of these systems, IR's effectiveness dramatically influences the overall performance and outcomes of RAG models. We call for papers that will seek to revisit and emphasize the fundamental principles underpinning RAG systems. At the end of the workshop, we aim to have a clearer understanding of how robust information retrieval mechanisms can significantly enhance the capabilities of RAG systems. The workshop will serve as a platform for experts, researchers, and practitioners. We intend to foster discussions, share insights, and encourage research that underscores the vital role of Information Retrieval in the future of generative systems.

This paper presents the project Comprehensive spAce wEather Studies for the ASPIS prototype Realization (CAESAR), which aims to tackle the relevant aspects of Space Weather (SWE) science and develop a prototype of the scientific data centre for Space Weather of the Italian Space Agency (ASI) called ASPIS (ASI SPace Weather InfraStructure). To this end, CAESAR involves the majority of the SWE Italian community, bringing together 10 Italian institutions as partners, and a total of 92 researchers. The CAESAR approach encompasses the whole chain of phenomena from the Sun to Earth up to planetary environments in a multidisciplinary, comprehensive, and unprecedented way. Detailed and integrated studies are being performed on a number of well-observed “target SWE events”, which exhibit noticeable SWE characteristics from several SWE perspectives. CAESAR investigations synergistically exploit a great variety of different products (datasets, codes, models), both long-standing and novel, that will be made available in the ASPIS prototype: this will consist of a relational database (DB), an interface, and a wiki-like documentation structure. The DB will be accessed through both a Web graphical interface and the ASPIS.py module, i.e., a library of functions in Python, which will be available for download and installation. The ASPIS prototype will unify multiple SWE resources through a flexible and adaptable architecture, and will integrate currently available international SWE assets to foster scientific studies and advance forecasting capabilities.