Vincenzo Lomonaco

Vincenzo Lomonaco

Auto-generated profile, based on publicly available information

Vincenzo Lomonaco

LUISS, Italy

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 August 2019 in Neural Networks
Verified
Journal Article
Closed access
DMDavide Maltoni
Davide Maltoni
VLVincenzo Lomonaco
Vincenzo Lomonaco

1 November 2021 in Neural Networks
Verified
Journal Article
Closed access
ACAndrea Cossu
Andrea Cossu
ACAntonio Carta
Antonio Carta
VLVincenzo Lomonaco
Vincenzo Lomonaco
DBDavide Bacciu
Davide Bacciu

1 July 2024 in Neural Networks
Verified
Journal Article
Open access
ACAndrea Cossu
Andrea Cossu
ACAntonio Carta
Antonio Carta
LPLucia Passaro
Lucia Passaro
VLVincenzo Lomonaco
Vincenzo Lomonaco
TTTinne Tuytelaars
Tinne Tuytelaars
DBDavide Bacciu
Davide Bacciu
Pre-trained models are commonly used in Continual Learning to initialize the model before training on the stream of non-stationary data. However, pre-training is rarely applied during Continual Learning. We investigate the characteristics of the Continual Pre-Training scenario, where a model is continually pre-trained on a stream of incoming data and only later fine-tuned to different downstream tasks. We introduce an evaluation protocol for Continual Pre-Training which monitors forgetting against a Forgetting Control dataset not present in the continual stream. We disentangle the impact on forgetting of 3 main factors: the input modality (NLP, Vision), the architecture type (Transformer, ResNet) and the pre-training protocol (supervised, self-supervised). Moreover, we propose a Sample-Efficient Pre-training method (SEP) that speeds up the pre-training phase. We show that the pre-training protocol is the most important factor accounting for forgetting. Surprisingly, we discovered that self-supervised continual pre-training in both NLP and Vision is sufficient to mitigate forgetting without the use of any Continual Learning strategy. Other factors, like model depth, input modality and architecture type are not as crucial.

1 February 2022 in Artificial Intelligence
Verified
Journal Article
Closed access
VLVincenzo Lomonaco
Vincenzo Lomonaco
LPLorenzo Pellegrini
Lorenzo Pellegrini
PRPau Rodriguez
Pau Rodriguez
MCMassimo Caccia
Massimo Caccia
QSQi She
Qi She
DMDavide Maltoni
Davide Maltoni

15 March 2023 in AI & SOCIETY
Verified
Journal Article
Open access
PDF file
Angelo TrottaAngelo Trotta
Angelo Trotta
MZMarta Ziosi
Marta Ziosi
VLVincenzo Lomonaco
Vincenzo Lomonaco

16 November 2023 in Nature Electronics
Verified
Journal Article
Closed access
PDF file
DKDhireesha Kudithipudi
Dhireesha Kudithipudi
ADAnurag Daram
Anurag Daram
AZAbdullah M. Zyarah
Abdullah M. Zyarah
FZFatima Tuz Zohora
Fatima Tuz Zohora
JAJames B. Aimone
James B. Aimone
BEBenjamin Epstein
Benjamin Epstein