2 July 2021 in Applied Intelligence
1 March 2022 in Chinese Journal of Electronics
29 May 2024 in Engineering Applications of Artificial Intelligence
5 August 2022 in Applied Intelligence
Pre-trained language models achieve high performance on machine reading comprehension task, but these models lack robustness and are vulnerable to adversarial samples. Most of the current methods for improving model robustness are based on data enrichment. However, these methods do not solve the problem of poor context representation of the machine reading comprehension model. We find that context representation plays a key role in the robustness of the machine reading comprehension model, dense context representation space results in poor model robustness. To deal with this, we propose a Multi-task machine Reading Comprehension learning framework via Contrastive Learning. Its main idea is to improve the context representation space encoded by the machine reading comprehension models through contrastive learning. This special contrastive learning we proposed called Contrastive Learning in Context Representation Space(CLCRS). CLCRS samples sentences containing context information from the context as positive and negative samples, expanding the distance between the answer sentence and other sentences in the context. Therefore, the context representation space of the machine reading comprehension model has been expanded. The model can better distinguish between sentence containing correct answers and misleading sentence. Thus, the robustness of the model is improved. Experiment results on adversarial datasets show that our method exceeds the comparison models and achieves state-of-the-art performance.
2 December 2022 in Human-Centric Intelligent Systems
Few-shot relation extraction is one of the current research focuses. The key to this research is to fully extract the relation semantic information through very little training data. Intuitively, raising the relation semantics awareness in sentences can improve the efficiency of the model to extract relation features to alleviate the overfitting problem in few-shot learning. Therefore, we propose an enhanced relation semantic feature model based on prototype network to extract relations from few-shot texts. Firstly, we design a multi-level embedding encoder with position information and Transformer, which uses local information in the text to enhance the relation semantics representation. Secondly, the encoded relation features are fed into the novel prototype network, which designs a method that utilizes query prototype-level attention to guide the extraction of supporting prototypes, thereby enhancing the prototypes representation to better classify the relations in query sentences. Finally, through experimental comparison and discussion, we prove and analyze the effectiveness of the proposed multi-level embedding encoder, and prototype-level attention can enhance the stability of the model. Furthermore, our model has substantial improvements over baseline methods.
9 June 2024 in Expert Systems with Applications