The extensive use of large language models (LLMs) across various natural language processing tasks has markedly elevated the intelligence of legal systems. Despite their exceptional performance in terms of accuracy, these systems still struggle with explainability. To tackle this challenge, we propose an approach to boost the question-answering abilities of LLMs through data synthesis, focusing on Qwen-7B. By incorporating Retrieval-Augmented Generation (RAG) techniques, we enhance the system’s transparency and reliability by introducing detailed reasoning processes (CoT Prompts). Our experimental results indicate that our trained LLMs exhibit significant improvements in both answer accuracy and explainability, especially in objective evaluation tasks. Additionally, subjective assessments reveal that the model’s responses are not only precise but also highly understandable, thus boosting user confidence in the system. Overall, our research offers valuable insights and technical advancements for the development of intelligent legal question-answering systems, with significant theoretical and practical implications.