Multimodal Entity Linking (MEL) is a crucial task that aims at linking ambiguous mentions within multimodal contexts to the referent entities in a multimodal knowledge base, such as Wikipedia. Existing methods focus heavily on using complex mechanisms and extensive model tuning methods to model the multimodal interaction on specific datasets. However, these methods overcomplicate the MEL task and overlook the visual semantic information, which makes them costly and hard to scale. Moreover, these methods cannot solve the issues like textual ambiguity, redundancy, and noisy images, which severely degrade their performance. Fortunately, the advent of Large Language Models (LLMs) with robust capabilities in text understanding and reasoning, particularly Multimodal Large Language Models (MLLMs) that can process multimodal inputs, provides new insights into addressing this challenge. However, how to design a universally applicable LLMs-based MEL approach remains a pressing challenge. To this end, we propose UniMEL, a unified framework which establishes a new paradigm to process multimodal entity linking tasks using LLMs. In this framework, we employ LLMs to augment the representation of mentions and entities individually by integrating textual and visual information and refining textual information. Subsequently, we employ the embedding-based method for retrieving and re-ranking candidate entities. Then, with only ~0.26% of the model parameters fine-tuned, LLMs can make the final selection from the candidate entities. Extensive experiments on three public benchmark datasets demonstrate that our solution achieves state-of-the-art performance, and ablation studies verify the effectiveness of all modules. Our code is available at https://github.com/Javkonline/UniMEL.

Multimodal Entity Linking (MEL) aims to address the ambiguity in multimodal mentions and associate them with Multimodal Knowledge Graphs (MMKGs). Existing works primarily focus on designing multimodal interaction and fusion mechanisms to enhance the performance of MEL. However, these methods still overlook two crucial gaps within the MEL task. One is the content discrepancy between mentions and entities, manifested as uneven information density. The other is the knowledge gap, indicating insufficient knowledge extraction and reasoning during the linking process. To bridge these gaps, we propose a novel framework FissFuse, as well as a plug-and-play knowledge-aware re-ranking method KAR. Specifically, FissFuse collaborates with the Fission and Fusion branches, establishing dynamic features for each mention-entity pair and adaptively learning multimodal interactions to alleviate content discrepancy. Meanwhile, KAR is endowed with carefully crafted instruction for intricate knowledge reasoning, serving as re-ranking agents empowered by Large Language Models (LLMs). Extensive experiments on two well-constructed MEL datasets demonstrate outstanding performance of FissFuse compared with various baselines. Comprehensive evaluations and ablation experiments validate the effectiveness and generality of KAR.

The stock movement prediction task has long been treated as one of the most crucial tasks for financial data mining. Unfortunately, prior arts may fail to capture the intricate nature of severe stock fluctuations, not to mention the difficulties in extracting reliable cues from massive, noisy financial news. In this case, it becomes more challenging to explore the distinctive evolving dynamics of individual stocks and discover deep-seated interstock dependencies within the market. To tackle these challenges, in this article, we propose a novel Memory-ENhanced Dependency Network, dubbed as MENDNet, for the multistock movement prediction task. Specifically, we first leverage the recent price movement from a dual-view perspective, providing a comprehensive depiction of each stock. Afterward, to mitigate noise information and the temporal effect of news cues, we assess their contribution based on the stock price volatility with a time-aware updating function. Along this line, we develop a well-elaborated memory structure to store selective news history for each stock, so that history embeddings can be dynamically estimated by attentively aggregating exclusive history information. Finally, we further delve into the evolving interdependencies among different stocks without relying on additional prior knowledge via a dependency-aware graph attention network. Extensive experiments on three real-world datasets validate the effectiveness of our proposed MENDNet framework compared with state-of-the-art baseline methods.