Multimodal entity linking (MEL) task, which aims at resolving ambiguous mentions to a multimodal knowledge graph, has attracted wide attention in recent years. Though large efforts have been made to explore the complementary effect among multiple modalities, however, they may fail to fully absorb the comprehensive expression of abbreviated textual context and implicit visual indication. Even worse, the inevitable noisy data may cause inconsistency of different modalities during the learning process, which severely degenerates the performance. To address the above issues, in this paper, we propose a novel Multi-GraIned Multimodal InteraCtion Network (MIMIC) framework for solving the MEL task. Specifically, the unified inputs of mentions and entities are first encoded by textual/visual encoders separately, to extract global descriptive features and local detailed features. Then, to derive the similarity matching score for each mention-entity pair, we device three interaction units to comprehensively explore the intra-modal interaction and inter-modal fusion among features of entities and mentions. In particular, three modules, namely the Text-based Global-Local interaction Unit (TGLU), Vision-based DuaL interaction Unit (VDLU) and Cross-Modal Fusion-based interaction Unit (CMFU) are designed to capture and integrate the fine-grained representation lying in abbreviated text and implicit visual cues. Afterwards, we introduce a unit-consistency objective function via contrastive learning to avoid inconsistency and model degradation. Experimental results on three public benchmark datasets demonstrate that our solution outperforms various state-of-the-art baselines, and ablation studies verify the effectiveness of designed modules.

The timely prediction of bankruptcy is highly desirable to guarantee an upward spiral for overall societal well-being. By extracting multifaceted information from the business interaction networks, Graph Neural Networks (GNNs) may be able to automatically make more informed predictions for bankruptcy, as compared to methods that rely heavily on abundant manpower to a large extent. Yet in real applications, bankruptcy prediction faces the key issue of quantity-imbalance: data usually comes with a long-tailed distribution wherein bankrupt corporates occupy the least of the data proportion but are our target to be identified. Apart from that, the topology-imbalance issue behind graph-structural data exacerbates prediction deterioration: feature propagation is dominated by non-bankrupt nodes through messages passing between nodes; thus, bankrupt nodes receive highly confusing information and could be easily assimilated by nearby non-bankrupt nodes. Unfortunately, the existing GNN methods are not immune to these two imbalance issues. To tackle the challenging but practically useful scenario, we propose a novel bankruptcy prediction model called the Quantity and Topology Imbalance-Aware Heterogeneous Graph Neural Network (QTIAH-GNN) to boost the final performance. Specifically, QTIAH-GNN employs the multi-hierarchy label-aware neighbor selection to conquer the topology-imbalance issue by using the class-semantic representation and the learnable parameterized similarity metric, and employs the imbalance-oriented loss to obtain the optimal tradeoff between the accuracies of the majority and minority classes. In experiments, we evaluate the proposed QTIAH-GNN on two large-scale, real-world datasets. The results show that QTIAH-GNN outperforms other state-of-the-art baselines in terms of prediction accuracy with superior efficiency and generalization ability, has stronger robustness to data imbalance, and provides meaningful model interpretation.

The prosperity of knowledge graphs, as well as related downstream applications, has raised the urgent need for knowledge graph completion techniques that fully support knowledge graph reasoning tasks, especially under the circumstance of training data scarcity. Although large efforts have been made on solving this challenge via few-shot learning tools, they mainly focus on simply aggregating entity neighbors to represent few-shot references, whereas the enhancement from latent semantic correlation within neighbors has been largely ignored. To that end, in this article, we propose a novel few-shot learning solution named SIM, a S emantic I nteraction M atching network that applies a Transformer framework to enhance the entity representation with capturing semantic interaction between entity neighbors. Specifically, we first design an entity-relation fusion module to adaptively encode neighbors with incorporating relation representation. Along this line, Transformer layers are integrated to capture latent correlation within neighbors, as well as the semantic diversification of the support set. Finally, a similarity score is attentively estimated with the attention mechanism. Extensive experiments on two public benchmark datasets demonstrate that our model outperforms a variety of state-of-the-art methods by a significant margin.