Julian von der Mosel

Julian von der Mosel

Auto-generated profile, based on publicly available information

Julian von der Mosel

Institute of Computer Science, University of Goettingen, Gttingen, GA, Germany

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
30 May 2022 in IEEE Transactions on Software Engineering
Verified
Journal Article
Closed access
JMJulian von der Mosel
Julian von der Mosel
ATAlexander Trautsch
Alexander Trautsch
SHSteffen Herbold
Steffen Herbold
Transformers are the current state-of-the-art of natural language processing in many domains and are using traction within software engineering research as well. Such models are pre-trained on large amounts of data, usually from the general domain. However, we only have a limited understanding regarding the validity of transformers within the software engineering domain, i.e., how good such models are at understanding words and sentences within a software engineering context and how this improves the state-of-the-art. Within this article, we shed light on this complex, but crucial issue. We compare BERT transformer models trained with software engineering data with transformers based on general domain data in multiple dimensions: their vocabulary, their ability to understand which words are missing, and their performance in classification tasks. Our results show that for tasks that require understanding of the software engineering context, pre-training with software engineering data is valuable, while general domain models are sufficient for general language understanding, also within the software engineering domain.