We present FinLLaMA-RAG, a Retrieval-Augmented Generation (RAG) framework built on LLaMA 3 to address complex document-level question answering. The model integrates dense retrieval, contextual fusion, and multi-hop reasoning to extract accurate, coherent answers from long and cross-referenced documents. Designed with high-stakes applications in mind, FinLLaMA-RAG supports automated tax compliance, financial fraud investigation, and regulatory reporting. Through joint optimization of retrieval and generation, it achieves strong results across several financial QA datasets. This work contributes to the development of explainable and reliable AI tools for financial and policy-oriented document analysis.
This study presents an exploratory methodological framework for examining structural changes in regulatory risk disclosure using sentence embeddings, multivariate anomaly detection, and explainable artificial intelligence. Prior research typically relies on dictionary-based word frequencies, tone indicators, or topic proportions to quantify risk disclosure. While these measures capture disclosure intensity, they do not directly assess whether the internal semantic organization of risk narratives has shifted relative to historical patterns. We propose a structural semantic deviation framework that represents each company–year disclosure using thematic shares and embedding-based dispersion statistics and evaluates deviations from a historical baseline through unsupervised anomaly detection. Using Item 1A Risk Factors from Wells Fargo and JPMorgan Chase surrounding the 2016 regulatory shock as a focused two-firm case study, we show that traditional lexical metrics do not clearly isolate structural breaks, whereas embedding-based semantic trajectories reveal substantial narrative reconfiguration. Isolation-based modeling provides stable and discriminative anomaly scores in this setting, and SHAP decomposition highlights semantic distance, litigation emphasis, and disclosure contraction as important drivers of deviation in 2025 out-of-sample disclosures. These findings should be interpreted as methodological evidence rather than broad population-level claims. The study demonstrates how structural semantic modeling can be operationalized in regulatory disclosure analysis and provides a transparent framework that can be extended to larger panels and cross-industry settings in future research.
This study examines structural changes in regulatory risk disclosure using a semantic modeling framework that integrates sentence embeddings, multivariate anomaly detection, and explainable artificial intelligence. Prior research typically relies on dictionary-based word frequencies, tone indicators, or topic proportions to quantify risk disclosure. While these measures capture disclosure intensity, they do not directly assess whether the internal semantic organization of risk narratives has shifted relative to historical patterns. We propose a structural semantic deviation framework that represents each company-year disclosure using thematic shares and embedding-based dispersion statistics, and evaluates deviations from a historical baseline through unsupervised anomaly detection. Using Item 1A Risk Factors from Wells Fargo and JPMorgan Chase surrounding the 2016 regulatory shock, we demonstrate that traditional lexical metrics fail to isolate structural breaks, whereas embedding-based semantic trajectories reveal substantial narrative reconfiguration. Isolation-based modeling provides stable and discriminative anomaly scores, and SHAP decomposition identifies semantic distance, litigation emphasis, and disclosure contraction as key drivers of deviation in 2025 out-of-sample disclosures. The results suggest that structural semantic modeling captures risk narrative transformation beyond word accumulation, offering an interpretable and scalable framework for regulatory risk assessment.
This paper presents a novel Retrieval-Augmented Generation (RAG) framework tailored for complex question answering tasks, addressing challenges in multi-hop reasoning and contextual understanding across lengthy documents. Built upon LLaMA 3, the framework integrates a dense retrieval module with advanced context fusion and multi-hop reasoning mechanisms, enabling more accurate and coherent response generation. A joint optimization strategy combining retrieval likelihood and generation cross-entropy improves the model's robustness and adaptability. Experimental results show that the proposed system outperforms existing retrieval-augmented and generative baselines, confirming its effectiveness in delivering precise, contextually grounded answers.
This paper presents a novel Retrieval-AugmentedGeneration (RAG) framework tailored for complex questionanswering tasks, addressing challenges in multi-hop reasoningand contextual understanding across lengthy documents. Builtupon LLaMA 3, the framework integrates a dense retrievalmodule with advanced context fusion and multi-hop reasoningmechanisms, enabling more accurate and coherent responsegeneration. A joint optimization strategy combining retrievallikelihood and generation cross-entropy improves the model’srobustness and adaptability. Experimental results show that theproposed system outperforms existing retrieval-augmented andgenerative baselines, confirming its effectiveness in deliveringprecise, contextually grounded answers.
This paper presents a novel Retrieval-AugmentedGeneration (RAG) framework tailored for complex questionanswering tasks, addressing challenges in multi-hop reasoningand contextual understanding across lengthy documents. Builtupon LLaMA 3, the framework integrates a dense retrievalmodule with advanced context fusion and multi-hop reasoningmechanisms, enabling more accurate and coherent responsegeneration. A joint optimization strategy combining retrievallikelihood and generation cross-entropy improves the model’srobustness and adaptability. Experimental results show that theproposed system outperforms existing retrieval-augmented andgenerative baselines, confirming its effectiveness in deliveringprecise, contextually grounded answers.