Strict adherence to data integrity and quality standards is crucial for the pharmaceutical industry to minimize undesired effects and ensure that medicines are of the required quality and safe for patients. A common data quality standard in the pharmaceutical industry is ALCOA+, which is a set of guiding principles for ensuring data integrity. Failure to comply with ALCOA+ guidelines, usually detected after audit inspections, may result in serious consequences for pharmaceutical manufacturers, such as the incurrence of fines, increase in costs, and production delays. It is, therefore, imperative to devise methods able to monitor ALCOA+ compliance and detect decreasing trends in data quality automatically. In this paper we present ALCOAi, a deep learning model based on the transformer architecture, which is able to process large quantities of non-homogeneous data and compute current and future ALCOA+ compliance. The proposed model can estimate trends concerning most ALCOA+ principles. The model was tested on a real dataset comprising raw sensor data, machine-provided values, and human-entered free-text data from two pharmaceutical manufacturing lines. The performed tests led to promising results in forecasting ALCOA+ compliance.

Large Language Models (LLMs) have emerged as a dominant paradigm in natural language processing, demonstrating strong performance across a wide range of generation and reasoning tasks. These systems depend on multi-stage training pipelines that integrate large-scale self-supervised pre-training, supervised fine-tuning, and alignment techniques. This paper presents a systematic mapping study of contemporary LLM training methodologies, emphasizing transformer-based architectures, optimization objectives, and data curation strategies as well as emerging sparse architectures such as Mixture-of-Experts (MoE) models. We analyze parameter-efficient fine-tuning approaches, retrieval-augmented generation frameworks, and multimodal training techniques, which we organize into a unified comparative taxonomy. We discuss key technical challenges such as scalability constraints, hallucination, bias amplification, and alignment–capability tradeoffs, then identify emerging research directions such as reasoning-centric training. This work provides a concise technical reference for researchers and practitioners working on scalable and reliable language model training.

We present a unified experimental decomposition of transformer prediction that separates information availability, native functional dependence, representational accessibility, and downstream predictive readout. For finite-vocabulary order-k Markov sources, the context penalty is exactly $H_L-H^{*}=I(X_{t+1};S_t\mid C_t^{\left(L\right)})$ and vanishes for $L\ge k$; all 15 sampled dense kernels show a strictly positive $H_{k-1}-H^{*}$ gap. Scratch transformers approach the corresponding context-limited optimum but retain optimization-dependent residuals. Post-training state-changing interventions steer the native predictive distribution toward the exact counterfactual oracle, and source-supported alternatives preserve 0.72–0.77-nat tracking gains while matched out-of-state controls remain near zero. In frozen Qwen2.5 representations, repeating the analysis over ten independent vocabulary maps per fixed checkpoint preserves the sharp state accessibility transition at $L=k$; Pythia-410M provides a cross-family replication. Capacity and sample-size controls show that residual predictive readout loss can arise even when the true state is supplied directly. Finally, a controlled hidden Markov model extension replaces the visible suffix state by a latent Bayesian belief state: exact filtering, scratch prediction, posterior probing, and predictive readout remain separately measurable, while the predictive gap to a 256-observation Bayesian reference decays smoothly rather than at a finite boundary. The results support a bounded four-level account of context sufficiency without equating decodability with native causal use.