Most automated program repair methods rely on test cases to determine the correctness of the generated patches. However, due to the incompleteness of available test suites, some patches that pass all the test cases may still be incorrect. This issue is known as the patch overfitting problem. Overfitting problem is a longstanding problem in automated program repair. Due to overfitting patches, the patches obtained by automated program repair tools require further validation to determine their correctness. Researchers have proposed many methods to automatically assess the correctness of patches, but no systematic review provides a detailed introduction to this problem, the existing solutions, and the challenges. To address this deficiency, we systematically review the existing approaches to patch correctness assessment. We first offer a few examples of overfitting patches to acquire a more detailed understanding of this problem. We then propose a comprehensive categorization of publicly available techniques and datasets, examine the commonly used evaluation metrics, and perform an in-depth analysis of the effectiveness of the existing models in addressing the challenge of overfitting. Based on our analysis, we provided the difficulties encountered by current methodologies, alongside the possible avenues for future research exploration.

Modern software systems continuously undergo code upgrades to enhance functionality, security, and performance, and Large Language Models (LLMs) have demonstrated remarkable capabilities in code migration tasks. However, while research on automated code migration which including refactoring, API adaptation, and dependency updates has advanced rapidly, the exploration of the automated environment interaction that must accompany it remains relatively scarce. In practice, code and its environment are intricately intertwined. Relying solely on static analysis of the environment leads to an inadequate understanding of the target setting, prolongs feedback cycles, and consequently causes significant rework and project delays, thereby reducing overall efficiency. We contend that successful software evolution demands a holistic perspective that integrates both code and environment migration. To understand the current landscape and challenges, we first provide an overview of the status of automated environment construction. We then propose a novel framework paradigm that tightly integrates automated environment setup with the code migration workflow. Finally, we explore the challenges and future directions for automated environment interaction within the code migration domain. Our findings emphasize that without automated environment interaction, the automation of code migration is only half complete.

Automated Program Repair (APR) is a crucial task in software development and maintenance, aiming to patch software bugs automatically without human intervention. The rise of Large Language Models (LLMs) has significantly advanced APR. However, as researchers delve deeper into APR as a downstream task for LLMs, a key challenge remains: how to effectively integrate APR-specific information to complement LLMs capabilities. This survey revisits 124 existing APR studies, most from 2021 to 2024, from two perspectives: Utilizing LLMs and APR-specific Information. First, we define the concept of APR-specific information. Then, we summarize techniques for utilizing LLMs in patch generation in four dimensions. Next, we explore critical factors influencing the effectiveness of generating patches by LLMs, such as prompting context and fine-tuning configurations. After that, we focus on the evaluation of generated patches, including benchmarks, reasoning cost scaling, etc. Furthermore, we distill all APR-specific information (e.g., bug-fix pairs and error messages), highlighting their unique importance and features. Finally, we comprehensively outline challenges, limitations, and potential future research directions in APR in the LLM era. To conclude, by adopting the two perspectives, we aim to provide valuable insights into mining and leveraging APR-specific information in utilizing LLMs process for the APR community.

Legal Judgment Prediction (LJP) focuses on predicting judgment results based on the facts of cases. While State-of-the-Art (SOTA) methods have shown impressive performance in law article prediction and charge prediction, they still exhibit weaknesses in prison term prediction. One major reason is that existing models fail to mimic human legal quantitative reasoning to understand monetary features in case facts. Consequently, they do not rigorously quantify the severity of the crime, which is essential for prison term prediction. In this article, we explore and explain how to leverage monetary features to improve LJP via quantitative reasoning. Specifically, we propose QR-LJP, a quantitative reasoning-based LJP model, to integrate legal reasoning knowledge into the prediction process. QR-LJP first employs a curated LLM to extract monetary values from case facts and uses legal quantitative reasoning logic to determine the total crime amount, serving as the quantitative measure of the crime’s severity. This measure is subsequently used to make judgment predictions. We evaluate our model on the real-world dataset CAIL-2018. Experimental results demonstrate that our model outperforms current SOTAs, highlighting the effectiveness of legal quantitative reasoning. Moreover, applying our quantitative reasoning strategy to existing SOTA methods yields significant improvements, especially in macro-F1 scores.