Modern organizations collect increasing volumes of data to drive decision-making, often stored in centralized repositories such as data lakes, which consist of diverse structured and unstructured datasets. However, these repositories often suffer from issues such as incomplete, inconsistent, and low-quality data, which hinder data-driven insights. Existing methods for data imputation, including statistical techniques and machine learning approaches, often rely heavily on large amounts of labeled data and domain-specific knowledge, making them labor-intensive and limited in handling semantic heterogeneity across data formats. To address these challenges, this study proposes a novel retrieval-augmented generation (RAG) framework for data imputation that effectively combines the strengths of retrieval mechanisms and large language models (LLMs). This approach constructs semantic-based indexes for heterogeneous data, employs a recall and re-ranking strategy to enhance retrieval relevance, and proposes a method to reduce inference cost while ensuring imputation quality. Extensive experiments on real-world datasets demonstrate that the proposed framework significantly outperforms machine learning and deep learning approaches.
Large language models (LLMs) have made rapid progress, yet adapting them to downstream scenarios still commonly relies on supervised fine-tuning (SFT). When downstream data exhibit a substantial distribution shift from the model's prior training distribution, SFT can induce catastrophic forgetting. To narrow this gap, data rewriting has been proposed as a data-centric approach that rewrites downstream training data prior to SFT. However, existing methods typically sample rewrites from a prompt-induced conditional distribution, so the resulting targets are not necessarily aligned with the model's natural QA-style generation distribution. Moreover, reliance on fixed templates can lead to diversity collapse. To address these issues, we cast data rewriting as a policy learning problem and learn a rewriting policy that better matches the backbone's QA-style generation distribution while preserving diversity. Since distributional alignment, diversity and task consistency are automatically evaluable but difficult to optimize end-to-end with differentiable objectives, we leverage reinforcement learning to optimize the rewrite distribution under reward feedback and propose an RL-based data-rewriting agent. The agent jointly optimizes QA-style distributional alignment and diversity under a hard task-consistency gate, thereby constructing a higher-quality rewritten dataset for downstream SFT. Extensive experiments show that our method achieves downstream gains comparable to standard SFT while reducing forgetting on non-downstream benchmarks by 12.34% on average. Our code is available at https://anonymous.4open.science/r/Patch-the-Prompt-Gap-4112 .