Medical text classification, as a fundamental medical natural language processing task, aims to identify the categories to which a short medical text belongs. Current research has focused on performing the medical text classification task using a pre-training language model through fine-tuning. However, this paradigm introduces additional parameters when training extra classifiers. Recent studies have shown that the “prompt-tuning” paradigm induces better performance in many natural language processing tasks because it bridges the gap between pre-training goals and downstream tasks. The main idea of prompt-tuning is to transform binary or multi-classification tasks into mask prediction tasks by fully exploiting the features learned by pre-training language models. This study explores, for the first time, how to classify medical texts using a discriminative pre-training language model called ERNIE-Health through prompt-tuning. Specifically, we attempt to perform prompt-tuning based on the multi-token selection task, which is a pre-training task of ERNIE-Health. The raw text is wrapped into a new sequence with a template in which the category label is replaced by a [UNK] token. The model is then trained to calculate the probability distribution of the candidate categories. Our method is tested on the KUAKE-Question Intention Classification and CHiP-Clinical Trial Criterion datasets and obtains the accuracy values of 0.866 and 0.861. In addition, the loss values of our model decrease faster throughout the training period compared to the fine-tuning. The experimental results provide valuable insights to the community and suggest that prompt-tuning can be a promising approach to improve the performance of pre-training models in domain-specific tasks.

Introduction There is a lack of objective, accurate, and convenient methods for classification diagnostic hypopigmented dermatoses (HD) and severity evaluation of vitiligo. To achieve an accurate and intelligent classification diagnostic model of HD and severity evaluation model of vitiligo using a deep learning-based method. Methods A total of 11,483 images from 4744 patients with HD were included in this study. An optimal diagnostic model was constructed by merging the squeeze-and-excitation (SE) module with the candidate model, its diagnostic efficiency was compared with that of 98 dermatologists. An objective severity evaluation indicator was proposed through weighting method and combined with a segmentation model to form a severity evaluation model, which was then compared with the assessments conducted by three experienced dermatologists using the naked eye. Results The improved diagnosis model SE_ResNet-18 outperformed the other 11 classic models with an accuracy of 0.9389, macro-specificity of 0.9878, and macro-f1 score of 0.9395, and outperformed the different categories of 98 dermatologists (P < 0.001). The weighted Kappa test indicated medium consistency between the Indicator_v and the VASI_change (K = 0.567, P < 0.05). The optimal segmented model, HR-Net, had 0.8421 mIOU. The model-based severity evaluation results were not significantly different among the three experienced dermatologists. Conclusions This study proposes an objective, accurate, and convenient hybrid model for diagnosing HD and evaluating the severity of vitiligo, providing a method for dermatologists especially in grassroots hospitals, and provides a foundation for telemedicine.

Relation extraction, a fundamental task in natural language processing, aims to extract entity triples from unstructured data. These triples can then be used to build a knowledge graph. Recently, pre-training models that have learned prior semantic and syntactic knowledge, such as BERT and ERNIE, have enhanced the performance of relation extraction tasks. However, previous research has mainly focused on sequential or structural data alone, such as the shortest dependency path, ignoring the fact that fusing sequential and structural features may improve the classification performance. This study proposes a concise approach using the fused features for the relation extraction task. Firstly, for the sequential data, we verify in detail which of the generated representations can effectively improve the performance. Secondly, inspired by the pre-training task of next-sentence prediction, we propose a concise relation extraction approach based on the fusion of sequential and structural features using the pre-training model ERNIE. The experiments were conducted on the SemEval 2010 Task 8 dataset and the results show that the proposed method can improve the F1 value to 0.902.