Fact verification datasets are typically constructed using crowdsourcing techniques due to the lack of text sources with veracity labels. However, the crowdsourcing process often produces undesired biases in data that cause models to learn spurious patterns. In this paper, we propose CrossAug, a contrastive data augmentation method for debiasing fact verification models. Specifically, we employ a two-stage augmentation pipeline to generate new claims and evidences from existing samples. The generated samples are then paired cross-wise with the original pair, forming contrastive samples that facilitate the model to rely less on spurious patterns and learn more robust representations. Experimental results show that our method outperforms the previous state-of-the-art debiasing technique by 3.6% on the debiased extension of the FEVER dataset, with a total performance boost of 10.13% from the baseline. Furthermore, we evaluate our approach in data-scarce settings, where models can be more susceptible to biases due to the lack of training data. Experimental results demonstrate that our approach is also effective at debiasing in these low-resource conditions, exceeding the baseline performance on the Symmetric dataset with just 1% of the original data.

Deep neural networks (DNNs) require a large amount of manually labeled training data to make significant achievements. However, manual labeling is laborious and costly. In this study, we propose a method for automatically generating training data and effectively using the generated data to reduce the labeling cost. The generated data (called "machine-labeled data") is generated using a bagging-based bootstrapping approach. However, using the machine-labeled data does not guarantee high performance because of errors in the automatic labeling. In order to reduce the impact of mislabeling, we applied a transfer learning approach. The effect of our proposed method was verified with two versions of DNN-based named entity recognition (NER) models: bidirectional LSTM-CRF and vanilla BERT. We conducted NER tasks in two languages (English and Korean). The proposed method results in average F1 scores of 78.87% (3.9% point improvement) with bidirectional LSTM-CRF and 82.08% (1% point improvement) with BERT on three Korean NER datasets. In English, the performance increased by an average of 0.45% points with the two DNN-based models. The proposed NER systems outperform the baseline systems in both languages without the need for additional manual labeling.

End-to-end neural network-based approaches have recently demonstrated significant improvements in natural language processing (NLP). However, in the NLP application such as assistant systems, NLP components are still processed to extract results using a pipeline paradigm. The pipeline-based concept has issues with error propagation. In Korean, morphological analysis and part-of-speech (POS) tagging step, incorrectly analyzing POS tags for a sentence containing spacing errors negatively affects other modules behind the POS module. Hence, we present a multi-task learning-based POS tagging neural model for Korean with word spacing challenges. When we apply this model to the Korean morphological analysis and POS tagging, we get findings that are robust to word spacing errors. We adopt syllable-level input and output formats, as well as a simple structure for ELECTRA and RNN-CRF models for multi-task learning, and we achieve a good performance 98.30 of F1, better than previous studies on the Sejong corpus test set.

Named entity recognition (NER) is the process of categorizing named entities in a given text that suffers from the lack of labeled corpora, which is a long-standing issue. Deep neural networks have been successfully applied to NER tasks. However, they require a large number of annotated data. Regardless of the number of data made available, annotation requires significant human effort, which is expensive and time-consuming. Moreover, collecting labeled data that reflect contemporary surrounding statuses requires exhaustive follow-up and incurs correspondingly higher costs. Current NERs typically focus on the supervised learning of hand-crafted data. The most well-known dataset for NER shared tasks, which was released at the 2003 Conference on Natural Language Learning, is used for basic training and evaluation. Although the data are qualified, the database has low coverage of timely material. In this paper, we illustrate methods for swiftly labeling up-to-date data via distant supervision. To tackle the difficulty of annotating contemporary written texts, we generate labeled data articles that reflect the latest issues. We evaluated the proposed methods with bidirectional long short-term memory conditional random-field architecture using static and contextualized embedding methods. Our proposed models perform higher than state-of-the-art methods with average F1-scores 3.09% better with weakly labeledWikipedia data and 3.47% better with Cable News Network data. When using the NER model with Flair embedding, our method shows 1.50 and 3.26% higher F1-scores with weakly labeled Wikipedia and news data, respectively. Qualitatively, the proposed model also performs better when extracting contemporary keywords.