Multimodal Entity Linking (MEL) is a crucial task that aims at linking ambiguous mentions within multimodal contexts to the referent entities in a multimodal knowledge base, such as Wikipedia. Existing methods focus heavily on using complex mechanisms and extensive model tuning methods to model the multimodal interaction on specific datasets. However, these methods overcomplicate the MEL task and overlook the visual semantic information, which makes them costly and hard to scale. Moreover, these methods cannot solve the issues like textual ambiguity, redundancy, and noisy images, which severely degrade their performance. Fortunately, the advent of Large Language Models (LLMs) with robust capabilities in text understanding and reasoning, particularly Multimodal Large Language Models (MLLMs) that can process multimodal inputs, provides new insights into addressing this challenge. However, how to design a universally applicable LLMs-based MEL approach remains a pressing challenge. To this end, we propose UniMEL, a unified framework which establishes a new paradigm to process multimodal entity linking tasks using LLMs. In this framework, we employ LLMs to augment the representation of mentions and entities individually by integrating textual and visual information and refining textual information. Subsequently, we employ the embedding-based method for retrieving and re-ranking candidate entities. Then, with only ~0.26% of the model parameters fine-tuned, LLMs can make the final selection from the candidate entities. Extensive experiments on three public benchmark datasets demonstrate that our solution achieves state-of-the-art performance, and ablation studies verify the effectiveness of all modules. Our code is available at https://github.com/Javkonline/UniMEL.

Mobile Crowdsensing (MCS), as a novel data acquisition paradigm in the Internet of Things (IoT), incentivizes a large number of participants to collaboratively sense data for providing real-time services and accomplishing complex sensing tasks to benefit society. However, a major challenge hindering the further development of MCS is the risk of privacy leakage of participant data. In this paper, an effective integration of Federated Learning (FL) with MCS is proposed. The classic Federated Averaging (FedAvg) algorithm is enhanced, and differential privacy (DP) is introduced to locally preserve the privacy of sensitive user data, referred to as DP-FAG. In DP-FAG, noise is applied to sensitive participant data prior to global model aggregation. Moreover, all sensitive training data is securely stored on participant devices, effectively addressing the trusted third-party issue and ensuring data confidentiality. We conduct extensive experimental analysis on image classification tasks to validate the soundness and effectiveness of our proposed methodology.

Table mining is a popular research field that involves complicated technologies, including information retrieval, data mining, visual and textual understanding and logical reasoning. With the emergence of Large Language Models (LLMs), the field has witnessed considerable advancements, presenting new paradigms for table understanding, extraction, and reasoning. In this survey, we conduct a comprehensive review of the literature on table mining with LLMs. We begin by introducing the fundamental overview of tabular data and possible challenges in LLM-based table mining. Specifically, we explore the challenges unique to this domain, such as heterogeneous table structures, contextual ambiguity, and domain-specific knowledge requirements. Then, we summarize representative tabular tasks in table preparation and mining, categorizing existing methods along dimensions including task scope, model architecture, and application scenarios. Next, we describe advanced LLM-based learning strategies in table mining, including foundation models and training-free methods. We further review studies of trustworthy LLM-based table mining and some domain-specific applications. Finally, we discuss prospects and future directions in the field of LLM-based table mining, including issues of generalization, interpretability, efficiency, etc . We hope this survey provides a comprehensive resource for researchers and practitioners, paving the way for further exploration. The repository is at: https://github.com/USTCAGI/Awesome-LLM-Table-Mining">https://github.com/USTCAGI/Awesome-LLM-Table-Mining .