Answer selection is a crucial subtask in the question answering (QA) system. Conventional avenues for this task mainly concentrate on developing linguistic tools that are limited in both performance and practicability. Answer selection approaches based on deep learning have been well investigated with the tremendous success of deep learning in natural language processing (NLP). However, the traditional neural networks employed in existing answer selection models, i.e., RNN or CNN, typically suffer from obtaining the global text information due to their operating mechanisms. The recent Transformer neural network is considered to be good at extracting the global information by employing only self-attention mechanism. Thus, in this paper, we design a Transformer-based neural network for answer selection, where we deploy a BiLSTM behind the Transformer to acquire both global information and sequential features in the question or answer sentence. Different from the original Transformer, our Transformer-based network focuses on sentence embedding rather than the seq2seq task. In addition, we employ a BiLSTM rather than utilizing the position encoding to incorporate sequential features as the universal Transformer does. Furthermore, we apply three aggregated strategies to generate sentence embeddings for question and answer, i.e., the weighted mean pooling, the max pooling and the attentive pooling, leading to three corresponding Transformer-based models, i.e., QA-TFWP, QA-TFMP and QA-TFAP, respectively. Finally, we evaluate our proposals on a popular QA dataset WikiQA. The experimental results demonstrate that our proposed Transformer-based answer selection models can produce a better performance compared with several competitive baselines. In detail, our models outperform the state-of-the-art baseline by up to 2.37%, 2.83% and 3.79% in terms of MAP, MRR and accuracy, respectively.

Job‐resume matching (JRM) is the core of online recruitment services for predicting the matching degree between a job post and a resume. Most of the existing methods for JRM achieve a promising performance by simplifying this task as a matching between the free‐text attributes in the job post and the resume. However, they neglect the contributions of the semistructured multivariate attributes such as education and salary, which will result in an unsuccessful prediction. To address this issue, we propose a novel approach to comprehensively explore the Internal and EXternal InTeractions for semistructured multivariate attributes in JRM, i.e., InEXIT. In detail, we first encode the key and the value of each attribute as well as its source into the same semantic space. Next, to explore the complex relationships among the multivariate attributes, we propose to hierarchically model the internal interactions among the multivariate attributes inside the job post and the resume, as well as the external interactions between the job post and the resume. In particular, a stepwise fusion mechanism is designed to respectively integrate the key embeddings and the source embeddings into the value embeddings so as to clearly indicate the key and the source of the value. Finally, we employ an aggregation matching layer to predict the matching degree. We quantify the improvements of InEXIT against the competitive baselines on a real‐world dataset, showing a general improvement of 4.28%, 4.10%, and 3.56% over the state‐of‐the‐art baseline in terms of AUC, accuracy, and F1 score, respectively.

Answer selection is an essential step in a question answering (QA) system. Traditional methods for this task mainly focus on developing linguistic features that are limited in practice. With the great success of deep learning method in distributed text representation, deep learning-based answer selection approaches have been well investigated, which mainly employ only one neural network, i.e., convolutional neural network (CNN) or long short term memory (LSTM), leading to failures in extracting some rich sentence features. Thus, in this paper, we propose a collaborative learning-based answer selection model (QA-CL), where we deploy a parallel training architecture to collaboratively learn the initial word vector matrix of the sentence by CNN and bidirectional LSTM (BiLSTM) at the same time. In addition, we extend our model by incorporating the sentence embedding generated by the QA-CL model into a joint distributed sentence representation using a strong unsupervised baseline weight removal (WR), i.e., the QA-CLWR model. We evaluate our proposals on a popular QA dataset, InsuranceQA. The experimental results indicate that our proposed answer selection methods can produce a better performance compared with several strong baselines. Finally, we investigate the models’ performance with respect to different question types and find that question types with a medium number of questions have a better and more stable performance than those types with too large or too small number of questions.

Chinese Medical Named Entity Recognition (Chinese-MNER) aims to identify potential entities and their categories from the unstructured Chinese medical text. Existing methods for this task mainly incorporate the dictionary knowledge on the basis of traditional BiLSTM-CRF or BERT architecture. However, the construction of high-quality dictionaries is typically time consuming and labor-intensive, which may also damage the robustness of NER models. What is more, the limited amount of annotated Chinese-MNER data can easily lead to the over-fitting problem while training. With the aim of dealing with the above problems, we put forward a BERT-BiLSTM-CRF model by integrating the part-of-speech (POS) tagging features and a Regularization method (BBCPR) for Chinese-MNER. In BBCPR, we first leverage a POS fusion layer to incorporate external syntax knowledge. Next, we design a novel REgularization mothod with Adversarial training and Dropout (READ) to improve the model robustness. Specifically, READ focuses on reducing the difference between the predictions of two sub-models through minimizing the bidirectional KL divergence between the adversarial output and original output distributions for the same sample. Comprehensive evaluations on two public data sets, namely, cMedQANER and cEHRNER from the Chinese Biomedical Language Understanding Evaluation benchmark (ChineseBLUE), demonstrate the superiority of our proposal in Chinese-MNER. In addition, ablation study shows that READ can effectively improve the model performance. Our proposal does well in exploring the technical terms and identifying the word boundary.

Text classification aims to assign predefined labels to unlabeled sentences, which tend to struggle in real-world applications when only a few annotated samples are available. Previous works generally focus on using the paradigm of meta-learning to overcome the classification difficulties brought by insufficient data, where a set of auxiliary tasks is given. Accordingly, prompt-based approaches are proposed to deal with the low-resource issue. However, existing prompt-based methods mainly focus on English tasks, which generally apply English pretrained language models that can not directly adapt to Chinese tasks due to structural and grammatical differences. Thus, we propose a prompt-based Chinese text classification framework that uses generated natural language sequences as hints, which can alleviate the classification bottleneck well in low-resource scenarios. In detail, we first design a prompt-based fine-tuning together with a novel pipeline for automating prompt generation in Chinese. Then, we propose a refined strategy for dynamically and selectively incorporating demonstrations into each context. We present a systematic evaluation for analyzing few-shot performance on a wide range of Chinese text classification tasks. Our approach makes few assumptions about task resources and expertise and therefore constitutes a powerful, task-independent approach for few-shot learning.