Recently, there has been growing educational interest in competency. Global organizations, such as the United Nations (UN) and Organization for Economic Co-operation and Development (OECD), which are leading the discourse on education reform, are undertaking the lead in spreading awareness regarding competency education. Since 2015, the number of published articles on competency education has been rapidly increasing. This paper aims to provide significant implications for creating a sustainable future of competency education. A topic modeling method was used to empirically analyze latent topics and international research trends in 26,532 articles published on competency-based education (CBE). As a result of the analysis, 15 topics were derived, including “approach to competency development.” In addition, five topics including “learning skills” and “teacher training” were found to be hot topics with the increasing article publication. The rapidly changing modern society is calling for a transformation in education. We hope that the results of this study paves the way for further research exploring new directions for education, such as competency education.

As an extension of the transformer architecture, the BERT model has introduced a new paradigm for natural language processing, achieving impressive results in various downstream tasks. However, high-performance BERT-based models—such as ELECTRA, ALBERT, and RoBERTa—suffer from limitations such as poor continuous learning capability and insufficient understanding of domain-specific documents. To address these issues, we propose the use of an attention mechanism to combine BERT-based models with neural topic models. Unlike traditional stochastic topic modeling, neural topic modeling employs artificial neural networks to learn topic representations. Furthermore, neural topic models can be integrated with other neural models and trained to identify latent variables in documents, thereby enabling BERT-based models to sufficiently comprehend the contexts of specific fields. We conducted experiments on three datasets—Movie Review Dataset (MRD), 20Newsgroups, and YELP—to evaluate our model’s performance. Compared to the vanilla model, the proposed model achieved an accuracy improvement of 1–2% for the ALBERT model in multiclassification tasks across all three datasets, while the ELECTRA model showed an accuracy improvement of less than 1%.