Multi-modal entity alignment (MMEA) aims to identify equivalent entities between two multi-modal knowledge graphs for integration. Unfortunately, prior arts have attempted to improve the interaction and fusion of multi-modal information, which have overlooked the influence of modal-specific noise and the usage of labeled and unlabeled data in semi-supervised settings. In this work, we introduce a Pseudo-label Calibration Multi-modal Entity Alignment (PCMEA) in a semi-supervised way. Specifically, in order to generate holistic entity representations, we first devise various embedding modules and attention mechanisms to extract visual, structural, relational, and attribute features. Different from the prior direct fusion methods, we next propose to exploit mutual information maximization to filter the modal-specific noise and to augment modal-invariant commonality. Then, we combine pseudo-label calibration with momentum-based contrastive learning to make full use of the labeled and unlabeled data, which improves the quality of pseudo-label and pulls aligned entities closer. Finally, extensive experiments on two MMEA datasets demonstrate the effectiveness of our PCMEA, which yields state-of-the-art performance.

The rapid growth of online data has made it very convenient for people to obtain information. However, it also leads to the problem of “information overload”. Therefore, how to detect hot events from the massive amount of information has always been a problem. With the development of multimedia platforms, event detection has gradually developed from traditional single modality detection to multi-modality detection and is receiving increasing attention. The goal of multi-modality event detection is to discover events from a huge amount of online data with different data structures, such as texts, images and videos. These data represent real-world events from different perspectives so that they can provide more information about an event. In addition, event evolution is also a meaningful research direction; it models how events change dynamically over time and has great significance for event analysis. This paper comprehensively reviews the existing research on event detection and evolution. We first give a series of necessary definitions of event detection and evolution. Next, we discuss the techniques of data representation for event detection, including textual, visual, and multi-modality content. Finally, we review event evolution under multi-modality data. Furthermore, we review several public datasets and compare their results. At the end of this paper, we provide a conclusion and discuss future work.

In knowledge base question answering (KBQA) systems, relation detection and entity recognition are two core components. However, since the relation detection in KBQA contains thousands of relations and this task always becomes a zero-shot learning task due to the relations in some test samples while they have not appeared in training data, relation detection is more difficult than entity recognition. In addition, previous studies only considered these two tasks separately and did not take full advantage of their correlation. This article proposes a novel relation and entity joint extraction framework, named Gated-Attention-based Joint Training Model (Ga-JTM), to integrate relation detection and entity recognition. In addition, to train the two models simultaneously, a knowledge-driven gated unit based on a multi-head attention mechanism is designed. It combines the knowledge graph embeddings and the current context semantic information to process relation detection and entity recognition tasks, respectively. The experiments are conducted on a single-relation dataset (SimpleQuestions) and a multiple-relation dataset (WebQSP), and the experimental results demonstrate that our Ga-JTM is superior to the state-of-the-art (SOTA) performance of relation detection and can improve the performance of entity recognition and entity linking. Finally, these improvements contribute to the SOTA performance in our KBQA system.

The Chinese NER task consists of two steps, first determining entity boundaries and then labeling them. Some previous work incorporating related words from pre-trained vocabulary into character-based models has been demonstrated to be effective. However, the number of words that characters can match in the vocabulary is large, and their meanings vary widely. It is unreasonable to concatenate all the matched words into the character's representation without making semantic distinctions. This is because words with different semantics also have distinct vectors by the distributed representation. Moreover, mutual information maximization (MIM) provides a unified way to characterize the correction between different granularity of embeddings, we find it can be used to enhance the features in our task. Consequently, this paper introduces a novel Chinese NER model named SSMI based on semantic similarity and MIM. We first match all the potential word boundaries of the input characters from the pre-trained vocabulary and employ BERT to segment the input sentence to get the segmentation containing these characters. After computing their cosine similarity, we obtain the word boundary with the highest similarity and the word group with similarity score larger than a specific threshold. Then, we concatenate the most relevant word boundaries with character vectors. We further calculate the mutual information maximization of group, character and sentence, respectively. Finally, we feed the result from the above steps to our novel network. The results on four Chinese public NER datasets show that our SSMI achieves state-of-the-art performance.