Recently, realistic services like virtual reality and augmented reality have gained popularity. These realistic services require deterministic transmission with end-to-end low latency and high reliability for practical applications. However, for these real-time services to be deterministic, the network core should provide the requisite level of network. To deliver differentiated services to each real-time service, network service providers can classify applications based on traffic. However, due to the presence of personal information in headers, application classification based on encrypted application data is necessary. Initially, we collected application traffic from four well-known applications and preprocessed this data to extract encrypted application data and convert it into model input. We proposed a lightweight transformer model consisting of an encoder, a global average pooling layer, and a dense layer to categorize applications based on the encrypted payload in a packet. To enhance the performance of the proposed model, we determined hyperparameters using several performance evaluations. We evaluated performance with 1D-CNN and ET-BERT. The proposed transformer model demonstrated good performance in the performance evaluation, with a classification accuracy and F1 score of 96% and 95%, respectively. The time complexity of the proposed transformer model was higher than that of 1D-CNN but performed better in application classification. The proposed transformer model had lower time complexity and higher classification performance than ET-BERT.

Self-supervised learning models, such as BERT, have improved the performance of various tasks in natural language processing. Although the effect is reduced in the out-of-domain field and not the the trained domain thus representing a limitation, it is difficult to train a new language model for a specific domain since it is both time-consuming and requires large amounts of data. We propose a method to quickly and effectively apply the pre-trained language models trained in the general domain to a specific domain’s vocabulary without re-training. An extended vocabulary list is obtained by extracting a meaningful wordpiece from the training data of the downstream task. We introduce curriculum learning, training the models with two successive updates, to adapt the embedding value of the new vocabulary. It is convenient to apply because all training of the models for downstream tasks are performed in one run. To confirm the effectiveness of the proposed method, we conducted experiments on AIDA-SC, AIDA-FC, and KLUE-TC, which are Korean classification tasks, and subsequently achieved stable performance improvement.

Curriculum learning is a method of prioritizing learning data to improve learning performance. In this paper, we propose a new algorithm that determines how to select learning data and when to start and stop curriculum learning by considering learning errors. We use entropy to select data samples with less consistent predictions and automatically determine the warming-up period based on the characteristics of the data. Additionally, to mitigate learning bias, we introduced a variable that adjusts the range of sample selection according to the progress of the training. To validate our method, we conducted extensive experiments on both balanced and imbalanced data classification tasks, and our proposed approach showed an average improvement of about 1.8%, with a maximum improvement of up to 4.4%, compared to previously suggested methods.

GPT (Generative Pre-trained Transformer) is a generative language model that demonstrates outstanding performance in the field of text generation. Generally, the attention mechanism of the transformer model behaves similarly to a copy distribution. However, due to the absence of a dedicated encoder, it is challenging to ensure that the input is retained for generation. We propose a model that emphasizes the copy mechanism in GPT. We generate masks for the input words to initialize the distribution and explicitly encourage copying through training. To demonstrate the effectiveness of our approach, we conducted experiments to restore ellipsis and anaphora in dialogue. In a single domain, we achieved 0.4319 (BLEU), 0.6408 (Rouge-L), 0.9040 (simCSE), and 0.9070 (BERTScore), while in multi-domain settings we obtained 0.4611 (BLEU), 0.6379 (Rouge-L), 0.8902 (simCSE), and 0.8999 (BERTScore). Additionally, we evaluated the operation of the copy mechanism on out-of-domain data, yielding excellent results. We anticipate that applying the copy mechanism to GPT will be useful for utilizing language models in constrained situations.

Class imbalance remains a significant challenge in classification, often leading to poor generalization on underrepresented classes. While Oversampling methods mitigate this issue by replicating minority class instances to balance class distributions, they typically overlook the informativeness of individual samples. In this paper, we propose an entropy-guided data selection strategy that dynamically prioritizes samples exhibiting frequent prediction changes during training, that is, those with high predictive entropy. Such uncertain samples are expected to contribute more effectively to the learning process. Moreover, we incorporate a credal set-based weighting scheme that adjusts class-wise selection probabilities according to global imbalance severity, quantified using the Gini coefficient. This adjustment penalizes overrepresented classes while increasing the sampling probability of rare but uncertain examples. Experiments on benchmark datasets show that the proposed method improves overall classification performance across imbalanced data settings, while also showing a more balanced trade-off across head, body, and tail classes.