Large language models trained on large-scale text corpora have become the primary solution for various natural language processing tasks. The alignment work to make the output content of large models consistent with human values and true intentions has become a significant research topic. This paper investigates the alignment of large models and related evaluation work by specifying the alignment goals at three levels: human instructions, human preferences, and human values. And then introducing the alignment paradigms and methods such as In-context Learning, Supervised Fine-tuning, and Reinforcement Learning, along with evaluation work derived from the alignment task, and presenting the existing evaluation paradigms and benchmarks. Finally, by synthesizing and refining existing research findings, the paper provides insights into the future development of alignment tasks for large language models.

Machine reading comprehension is a crucial area of research in natural language processing, aiming to enable machines to read and understand text as humans do, and to answer questions related to the text content. The pre-trained language model, represented by BERT, has achieved superior results compared to traditional models in many NLP tasks, leading to the rise of the pre-trained paradigm in the field of natural language processing. This paper addresses the issue that the pre-trained language model lacks the ability to extract long-distance semantic relations and make efficient use of linguistic features. Firstly, the recent developments of the pre-trained language model are described. Then, two feature maps are used to intuitively express the structured long-distance semantic correlation features, and the traditional sequence structure features are integrated. The influence of different feature map construction methods on machine reading comprehension is compared. Finally, the application of the pre-trained language model in machine reading comprehension is summarized and prospected. By fusing graph structure features, the model can learn more abundant language knowledge, thus further improving its reasoning ability.

Large language models (LLMs) have achieved remarkable progress in natural language processing, yet their ability to perform complex logical reasoning remains limited. Existing approaches such as prompting, retrieval-augmented generation, and parameter-efficient fine-tuning (PEFT) provide partial improvements but often suffer from prompt sensitivity, semantic compression, or additional computational cost. In this work, we propose a Residual Feature Enhancement (RFE) module, a lightweight architectural component designed to strengthen reasoning ability while maintaining computational efficiency. RFE integrates a dimension-preserving linear transformation, SwiGLU nonlinear activation, and residual connections to enrich attention outputs without altering the backbone structure. We conducted comprehensive experiments across six reasoning and comprehension benchmarks—LogiQA, ReClor, LogiQA2.0, GSM8K, HellaSwag, and MBPP—covering deductive reasoning, standardized test comprehension, commonsense inference, and program synthesis. Results demonstrate that ChatGLM4-9B augmented with RFE consistently achieves superior performance compared with both adapter-based methods and larger-scale baselines. Specifically, ChatGLM4-9B+RFE attains 68.20% on LogiQA, 82.00% on ReClor, 79.74% on LogiQA 2.0, 95.68% on GSM8K, 72.42% on HellaSwag, and 56.82% on MBPP, all of which surpass the Adapter mechanism (67.68%, 81.15%, 78.52%, 94.47%, 66.85%, 55.02%) and show clear advantages over open-source baselines such as Qwen1.5-MoE-A2.7B, Llama3.1-8B, and DeepSeek distilled models. Ablation studies further confirm that removing RFE leads to performance degradation of up to 3.84 percentage points, and convergence analysis shows improved stability and faster training.

To address the problems of insufficient semantic fusion between text and questions and the lack of consideration of global semantic information encountered in machine reading comprehension models, we proposed a machine reading comprehension model called BERT_hybrid based on BERT and hybrid attention mechanism. In this model, BERT is utilized to separately map the text and questions into the feature space. Through the integration of Bi-LSTM, attention mechanism, and self-attention mechanism, the proposed model achieves comprehensive semantic fusion between text and questions. The probabilities distribution of answers is computed using Softmax. Experimental results on the public dataset DuReader demonstrate that the proposed model achieves improvements in BLEU-4 and ROUGE-L scores compared to existing models. Furthermore, to validate the effectiveness of the proposed model design, we analyze the factors influencing the model’s performance.

Semantic Role Labeling (SRL) aims to recover predicate‐argument structures in a sentence. Although character‐based pretrained language models perform strongly on Chinese SRL, they still suffer from core‐argument boundary attribution errors, especially around function words and tightly coupled predicate‐argument patterns. To address this, we propose RoBERTa‐MPBF‐CRF, a feature‐fusion architecture that augments a Chinese RoBERTa encoder with character‐level part‐of‐speech (POS) features, introduces a pooling‐enhanced biaffine scorer to model predicate‐token interactions, and uses a CRF decoder to enforce valid BIOES label transitions. We evaluate on Chinese PropBank (CPB) under the gold‐predicate setting and use automatically predicted POS tags (LTP) at test time. Our full model achieves an F1 score of 90.89% on CPB and improves substantially over a strong RoBERTa + POS + CRF baseline. Ablation results show that the pooling‐enhanced biaffine scorer and POS features contribute +2.09 and +1.55 absolute F1, respectively. We further report a cross‐corpus evaluation on Chinese OntoNotes to assess transfer. Overall, these results indicate that combining lightweight pooling, explicit POS cues, and structured biaffine scoring can improve Chinese SRL while keeping the evaluation protocol explicit and reproducible.

Semantic Role Labeling (SRL) serves as the foundational and pivotal technology for semantic analysis. However, methods relying on pre-trained language models are constrained by issues such as semantic ambiguity and training complexity. Addressing this concern, this paper proposes a Chinese SRL approach that integrates pre-trained language models with Biaffine technology, aiming to enhance the model’s capability in processing semantic information from long sentences while reducing training complexity. By incorporating pooling techniques and part-of-speech features, the model exhibits significant improvements in capturing semantic role boundary relationships. Experimental results demonstrate that the RoBERTa-MPBF model employing maximum pooling achieves an F1 score of 90.89% on the CPB dataset, outperforming models solely based on conditional random fields. Moreover, the introduction of part-of-speech tagging results in an average F1 score improvement of approximately 1.5%. Despite the increased computational burden, considering the performance enhancement, this additional time cost is deemed acceptable. In convolutional kernel size testing, the model maintains F1 scores between 88.6% and 88.8% when the kernel size is 2 or 3. However, as the kernel size increases to 4, the F1 score drops to 80.37%, and further increases to 5 result in an F1 score reduction to 69.51%.