So far, discontinuous named entity recognition (NER) has received increasing research attention and many related methods have surged such as hypergraph-based methods, span-based methods, and sequence-to-sequence (Seq2Seq) methods, etc. However, these methods more or less suffer from some problems such as decoding ambiguity and efficiency, which limit their performance. Recently, grid-tagging methods, which benefit from the flexible design of tagging systems and model architectures, have shown superiority to adapt for various information extraction tasks. In this paper, we follow the line of such methods and propose a competitive grid-tagging model for discontinuous NER. We call our model TOE because we incorporate two kinds of Tag-Oriented Enhancement mechanisms into a state-of-the-art (SOTA) grid-tagging model that casts the NER problem into word-word relationship prediction. First, we design a Tag Representation Embedding Module (TREM) to force our model to consider not only word-word relationships but also word-tag and tag-tag relationships. Concretely, we construct tag representations and embed them into TREM, so that TREM can treat tag and word representations as queries/keys/values and utilize self-attention to model their relationships. On the other hand, motivated by the Next-Neighboring-Word (NNW) and Tail-Head-Word (THW) tags in the SOTA model, we add two new symmetric tags, namely Previous-Neighboring-Word (PNW) and Head-Tail-Word (HTW), to model more fine-grained word-word relationships and alleviate error propagation from tag prediction. In the experiments of three benchmark datasets, namely CADEC, ShARe13 and ShARe14, our TOE model pushes the SOTA results by about 0.83%, 0.05% and 0.66% in F1, demonstrating its effectiveness.
16 April 2024 in IEEE Transactions on Knowledge and Data Engineering
Few-shot named entity recognition (NER) exploits limited annotated instances to identify named mentions. Effectively transferring the internal or external resources thus becomes the key to few-shot NER. While the existing prompt tuning methods have shown remarkable few-shot performances, they still fail to make full use of knowledge. In this work, we investigate the integration of rich knowledge to prompt tuning for stronger few-shot NER. We propose incorporating the deep prompt tuning framework with threefold knowledge (namely TKDP ), including the internal 1) context knowledge and the external 2) label knowledge & 3) sememe knowledge . TKDP encodes the three feature sources and incorporates them into soft prompt embeddings, which are further injected into an existing pre-trained language model to facilitate predictions. On five benchmark datasets, the performance of our knowledge-enriched model was boosted by at most 11.53% F1 over the raw deep prompt method, and it significantly outperforms 9 strong-performing baseline systems in 5-/10-/20-shot settings, showing great potential in few-shot NER. Our TKDP framework can be broadly adapted to other few-shot tasks without much effort.
11 September 2025 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Multimodal information extraction (IE) tasks have attracted increasing attention because many studies have shown that multimodal information benefits text information extraction. However, existing multimodal IE datasets mainly focus on sentence-level image-facilitated IE in English text, and pay little attention to video-based multimodal IE and fine-grained visual grounding. Therefore, in order to promote the development of multimodal IE, we constructed a multimodal multilingual multitask dataset, named M$^{3}$D, which has the following features: (1) It contains paired document-level text and video to enrich multimodal information; (2) It supports two widely-used languages, namely English and Chinese; (3) It includes more multimodal IE tasks such as entity recognition, entity chain extraction, relation extraction and visual grounding. In addition, our dataset introduces an unexplored theme, i.e., biography, enriching the domains of multimodal IE resources. To establish a benchmark for our dataset, we propose an innovative hierarchical multimodal IE model. This model effectively leverages and integrates multimodal information through a Denoised Feature Fusion Module (DFFM). Furthermore, in non-ideal scenarios, modal information is often incomplete. Thus, we designed a Missing Modality Construction Module (MMCM) to alleviate the issues caused by missing modalities. Our model achieved an average performance of 53.80% and 53.77% on four tasks in English and Chinese datasets, respectively, which set a reasonable standard for subsequent research. In addition, we conducted more analytical experiments to verify the effectiveness of our proposed module. We believe that our work can promote the development of the field of multimodal IE.
Addressing the challenges of dense conceptual content and intricate knowledge relations in computer science textbooks, where traditional pipeline-based information extraction suffers from error propagation and semantic decoupling, this paper proposes a concept-driven joint extraction method termed CDE (Concept-Driven Extraction).First, the model’s ability to focus on domain-specific terminology is enhanced through conceptual priors and attention re-weighting. This is integrated with a predefined schema and structured instruction templates to achieve normalized output for both entities and relations. Second, efficient domain knowledge transfer for computer science textbooks is realized by performing Low-Rank Adaptation (LoRA) fine-tuning on the Qwen3-4B large language model. Finally, the construction of the computer science textbook knowledge graph is accomplished using the Neo4j graph database. On a self-constructed instruction dataset of computer science textbooks, CDE achieves an F1 score of 81.83%, representing an improvement of approximately 2.47 percentage points over the LKD-KGC baseline. This performance significantly surpasses that of traditional pipeline models and existing joint extraction approaches. Experimental results demonstrate that CDE can effectively improve knowledge extraction accuracy in the textbook domain, thereby providing a novel research avenue for the rapid construction of knowledge graphs for computer science educational materials.
Few-shot named entity recognition (NER) exploits limited annotated instances to identify named mentions. Effectively transferring the internal or external resources thus becomes the key to few-shot NER. While the existing prompt tuning methods have shown remarkable few-shot performances, they still fail to make full use of knowledge. In this work, we investigate the integration of rich knowledge to prompt tuning for stronger few-shot NER. We propose incorporating the deep prompt tuning framework with threefold knowledge (namely TKDP), including the internal 1) context knowledge and the external 2) label knowledge & 3) sememe knowledge. TKDP encodes the three feature sources and incorporates them into the soft prompt embeddings, which are further injected into an existing pre-trained language model to facilitate predictions. On five benchmark datasets, our knowledge-enriched model boosts by at most 11.53% F1 over the raw deep prompt method, and significantly outperforms 8 strong-performing baseline systems in 5-/10-/20-shot settings, showing great potential in few-shot NER. Our TKDP can be broadly adapted to other few-shot tasks without effort.
Multimodal knowledge graph completion (MMKGC) aims to predict missing entities by exploiting structural, visual, and textual information and is important for semantic retrieval, recommendation, and intelligent question answering. Existing relation-aware methods usually use relational context to adjust modality weights or fuse multimodal scores but rarely exploit the historical answer distribution of each directed relation as explicit ranking evidence. To address this limitation, we propose RAPC, a Relation Answer Space Prototype Calibration framework. For each query, RAPC obtains multimodal prior scores from structural, visual, textual, and image–text cross-modal branches, retrieves the top-k query-relevant anchors from the historical answer space of the corresponding directed relation, and aggregates them into prototype evidence. This evidence is injected through a prior-preserving selective calibration mechanism, while relation-aware hard negative training improves discrimination between true answers and similar false candidates. Experiments on DB15K, MKG-W, and MKG-Y show that RAPC achieves clear gains on DB15K and MKG-W and obtains the best MRR and Hits@1 on MKG-Y. Compared with the best external results, RAPC improves MRR and Hits@1 by 2.64 and 3.57 percentage points on DB15K and by 3.02 and 3.72 percentage points on MKG-W. These results show that relation-level historical answer distributions provide useful explicit evidence for top-ranked entity prediction.