2 January 2025 in Scientific Reports
Deep language models (DLMs) have exhibited remarkable language understanding and generation capabilities, prompting researchers to explore the similarities between their internal mechanisms and human language cognitive processing. This study investigated the representational similarity (RS) between the abstractive summarization (ABS) models and the human brain and its correlation to the performance of ABS tasks. Specifically, representational similarity analysis (RSA) was used to measure the similarity between the representational patterns (RPs) of the BART, PEGASUS, and T5 models’ hidden layers and the human brain’s language RPs under different spatiotemporal conditions. Layer-wise ablation manipulation, including attention ablation and noise addition was employed to examine the hidden layers’ effect on model performance. The results demonstrate that as the depth of hidden layers increases, the models’ text encoding becomes increasingly similar to the human brain’s language RPs. Manipulating deeper layers leads to more substantial decline in summarization performance compared to shallower layers, highlighting the crucial role of deeper layers in integrating essential information. Notably, the study confirms the hypothesis that the hidden layers exhibiting higher similarity to human brain activity play a more critical role in model performance, with their correlations reaching statistical significance even after controlling for perplexity. These findings deepen our understanding of the cognitive mechanisms underlying language representations in DLMs and their neural correlates, potentially providing insights for optimizing and improving language models by aligning them with the human brain’s language-processing mechanisms.
29 May 2025 in IEEE Transactions on Audio, Speech and Language Processing
Improving factual consistency in abstractive summarization has been a focus of recent research. One promising approach is the post-editing method. However, previous works have yet to make sufficient use of factual factors in summaries and suffer from the negative effect of the training datasets. In this paper, we first propose a novel factual error correction model FactCloze based on a conditional-generation cloze task. FactCloze can construct the causality among factual factors while being able to determine whether the blank can be answered. Then, we propose a data distillation method to generate a more faithful summarization dataset SummDSC via multiple-dimensional evaluation. We validate our method on both non-LLM and LLM-generated datasets. Besides BART and T5, we implement FactCloze using DeepSeek prompt. Finally, we examine the differences between LLM-based and traditional evaluation metrics for factual error correction.
11 February 2026 in Biomedical Signal Processing and Control
30 March 2026 in Neural Networks