Marco Cognetta

Mr. Marco Cognetta

orcid

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 January 2024
Verified
Conference Paper
Open access
PDF file
MCMarco Cognetta
Marco Cognetta
THTatsuya Hiraoka
Tatsuya Hiraoka
RSRico Sennrich
Rico Sennrich
Yuval PinterYuval Pinter
Yuval Pinter
NONaoaki Okazaki
Naoaki Okazaki
Marco Cognetta, Tatsuya Hiraoka, Rico Sennrich, Yuval Pinter, Naoaki Okazaki. Proceedings of the Fifth Workshop on Insights from Negative Results in NLP. 2024.

1 January 2024
Verified
Conference Paper
Open access
PDF file
MCMarco Cognetta
Marco Cognetta
VZVilém Zouhar
Vilém Zouhar
NONaoaki Okazaki
Naoaki Okazaki
Marco Cognetta, Vilém Zouhar, Naoaki Okazaki. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

1 January 2023
Verified
Conference Paper
Open access
PDF file
MCMarco Cognetta
Marco Cognetta
SMSangwhan Moon
Sangwhan Moon
LWLawrence Wolf-Sonkin
Lawrence Wolf-Sonkin
NONaoaki Okazaki
Naoaki Okazaki
Marco Cognetta, Sangwhan Moon, Lawrence Wolf-sonkin, Naoaki Okazaki. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023.

30 March 2024 in arXiv
Verified
Preprint
Open access
PDF file
MCMarco Cognetta
Marco Cognetta
THTatsuya Hiraoka
Tatsuya Hiraoka
NONaoaki Okazaki
Naoaki Okazaki
RSRico Sennrich
Rico Sennrich
Yuval PinterYuval Pinter
Yuval Pinter
We explore threshold vocabulary trimming in Byte-Pair Encoding subword tokenization, a postprocessing step that replaces rare subwords with their component subwords. The technique is available in popular tokenization libraries but has not been subjected to rigorous scientific scrutiny. While the removal of rare subwords is suggested as best practice in machine translation implementations, both as a means to reduce model size and for improving model performance through robustness, our experiments indicate that, across a large space of hyperparameter settings, vocabulary trimming fails to improve performance, and is even prone to incurring heavy degradation.

21 October 2024 in arXiv
Verified
Preprint
Open access
PDF file
MCMarco Cognetta
Marco Cognetta
NONaoaki Okazaki
Naoaki Okazaki
Tokenization is the first step in modern neural language model pipelines where an input text is converted to a sequence of subword tokens. We introduce from first principles a finite-state transduction framework which can efficiently encode all possible tokenizations of a regular language. We then constructively show that Byte-Pair Encoding (BPE) and MaxMatch (WordPiece), two popular tokenization schemes, fit within this framework. For BPE, this is particularly surprising given its resemblance to context-free grammar and the fact that it does not tokenize strings from left to right. An application of this is to guided generation, where the outputs of a language model are constrained to match some pattern. Here, patterns are encoded at the character level, which creates a mismatch between the constraints and the model's subword vocabulary. While past work has focused only on constraining outputs without regard to the underlying tokenization algorithm, our framework allows for simultaneously constraining the model outputs to match a specified pattern while also adhering to the underlying tokenizer's canonical tokenization.

1 January 2025
Verified
Conference Paper
Open access
PDF file
JLJunyoung Lee
Junyoung Lee
MCMarco Cognetta
Marco Cognetta
SMSangwhan Moon
Sangwhan Moon
NONaoaki Okazaki
Naoaki Okazaki
Junyoung Lee, Marco Cognetta, Sangwhan Moon, Naoaki Okazaki. Proceedings of the Eighth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2025). 2025.