Low-resource languages remain underserved by contemporary large language models (LLMs) because they lack sizable corpora, bespoke preprocessing tools, and the computing budgets assumed by mainstream alignment pipelines. Focusing on Kazakh, we present a 1.94B parameter LLaMA-based model that demonstrates how strong, culturally aligned performance can be achieved without massive infrastructure. The contribution is threefold. (i) Data and tokenization—we compile a rigorously cleaned, mixed-domain Kazakh corpus and design a tokenizer that respects the language’s agglutinative morphology, mixed-script usage, and diacritics. (ii) Training recipe—the model is built in two stages: causal language modeling from scratch followed by instruction tuning. Alignment is further refined with Direct Preference Optimization (DPO), extended by contrastive and entropy-based regularization to stabilize training under sparse, noisy preference signals. Two complementary resources support this step: ChatTune-DPO, a crowd-sourced set of human preference pairs, and Pseudo-DPO, an automatically generated alternative that repurposes instruction data to reduce annotation cost. (iii) Evaluation and impact—qualitative and task-specific assessments show that targeted monolingual training and the proposed DPO variant markedly improve factuality, coherence, and cultural fidelity over baseline instruction-only and multilingual counterparts. The model and datasets are released under open licenses, offering a reproducible blueprint for extending state-of-the-art language modeling to other under-represented languages and domains.
This study is devoted to the transcription of human speech in the Kazakh language in dynamically changing conditions. It discusses key aspects related to the phonetic structure of the Kazakh language, technical considerations in collecting the transcribed audio corpus, and the use of deep neural networks for speech modeling. A high-quality decoded audio corpus was collected, containing 554 h of data, giving an idea of the frequencies of letters and syllables, as well as demographic parameters such as the gender, age, and region of residence of native speakers. The corpus contains a universal vocabulary and serves as a valuable resource for the development of modules related to speech. Machine learning experiments were conducted using the DeepSpeech2 model, which includes a sequence-to-sequence architecture with an encoder, decoder, and attention mechanism. To increase the reliability of the model, filters initialized with symbol-level embeddings were introduced to reduce the dependence on accurate positioning on object maps. The training process included simultaneous preparation of convolutional filters for spectrograms and symbolic objects. The proposed approach, using a combination of supervised and unsupervised learning methods, resulted in a 66.7% reduction in the weight of the model while maintaining relative accuracy. The evaluation on the test sample showed a 7.6% lower character error rate (CER) compared to existing models, demonstrating its most modern characteristics. The proposed architecture provides deployment on platforms with limited resources. Overall, this study presents a high-quality audio corpus, an improved speech recognition model, and promising results applicable to speech-related applications and languages beyond Kazakh.
Large language models can emit fluent text while leaving intermediate semantic structure implicit. We study whether explicit event-role and logical-primitive side-state can improve a pretrained decoder without damaging its language behavior. We introduce PAT-ER, a decoder architecture with a normal token stream, an event-role register stream, and a primitive register stream. The primitive stream is motivated by the view that logical primitives answer characteristic semantic questions, such as what licenses a conclusion, what conflicts with it, or why evidence is insufficient. Across eight seeds on the same Qwen3-0.6B backbone, replacing token-pooled auxiliary heads with typed PAT-ER registers improves primitive macro-F1 by 0.209 (95% CI [0.182, 0.237]) and role-to-primitive macro-F1 by 0.091 (95% CI [0.074, 0.110]) with no language-model loss cost. A generic-register control shows that this is not merely the effect of adding latent registers: typed PAT-ER improves over generic registers by 0.116 primitive macro-F1 and 0.110 role-to-primitive macro-F1, with both confidence intervals excluding zero. A warm-started model then recovers pretrained language quality (LM loss 1.344 versus 2.555 for the frozen-backbone register model) while retaining most side-state behavior. Finally, a decoupled interface mode produces robust schema-grounded function calls on 242 held-out prompts (Hermes parse 0.952, exact arguments 0.981, JSON validity 1.000, IDK F1 1.000) while base-mode side-state metrics remain byte-identical to the warm-start baseline. The model is not a theorem prover and does not achieve perfect unseen tool-name copying; the contribution is a measured architecture signal and a usable, guarded interface.
Frozen speech-to-LLM systems train an adapter between a frozen audio encoder and a text LLM. We test whether adapters preserve sentence stress beyond transcripts and whether the LLM uses it. In a five-seed Qwen3-8B/WavLM baseline, transcript alignment lowers linear adapter Probe-K below the text-only baseline (0.211 vs. 0.290). The R1.8 configuration improves MLP-2 Probe-K from 0.245 to 0.306 (paired ) and passes three controls, but because warmup and augmentation also change, the gain is not attributed to alone; its crossing of the linear floor is neither statistically established nor capacity-matched. Response Probe-G remains 0.512 in both cohorts; single-seed LoRA and styled-teacher pilots do not improve it. A post hoc trace through all 36 Qwen blocks finds persistent absolute stress decodability in speech-slot states, but no reliable R1.8-over-R0 advantage at any state and no speech-conditioned answer margin; an explicit text tag instead yields a -nat margin. Attention differences depend on slot-length normalization, and late left/system concentration is shared across modalities rather than speech-specific. Thus, the tested system remains an end-to-end negative: adapter decodability does not imply causal response use. Scope is limited to sentence stress, mostly synthetic voices, one encoder, and one LLM.
Task failure alone cannot distinguish limited robot execution from incorrect target selection. We evaluate released SmolVLA and policies in VLABench using family-matched native controls, same-state trusted and null instructions, online symmetric stopping and a separate retention check. The main experiment contains 1860 attempts under three inference seeds; a command-duration follow-up adds 420. Native completion rises from 0/210 and 1/210 at 6 simulated seconds to 73/210 and 101/210 at 20 s, respectively, correcting a short evaluation cutoff. Reviewer-requested secondary analysis of 40 benchmark-derived paired scenes gives a symmetric request-swap contrast of 10.00 percentage points for (scene-bootstrap 95% CI [3.33, 17.08]) and zero for SmolVLA ([−4.17, 4.58]), yet requested choice under switched instructions is only 20/47 (42.6%) and 9/22 (40.9%) among goal-reaching attempts, respectively. Across all 60 nominal fixtures, the advantage over blank input decomposes into 11.39 points of coverage and 3.33 points of net-choice contribution ([1.11, 5.69]). Its shuffled comparison warrants caution: the original adjusted is borderline; 10/40 benchmark donors name a candidate, and all 20 constructed donors equal request A. Holding each command for two environment steps rather than one reduces native completion to 39/210 and 55/210 at the same 20 s budget while also changing command count and replanning time. The results demonstrate configuration-sensitive execution and a bounded instruction effect, not reliable target switching, architecture superiority or sim-to-real transfer.
LLM agents increasingly call tools and take irreversible actions, yet the reasoning that licenses an action is rarely checked before execution. We present Proof-Carrying Agent Actions (PCAA) : each governed action carries a typed Action Certificate that a planner-independent checker must accept before execution. A trusted execution shim derives the obligation set and the irreversibility classification from an authoritative tool contract and binds the certificate to the exact invocation with a canonical SHA-256 commitment. The checker compares that context with the certificate, re-derives typed support (proof, belief, hypothesis, contingency, conflict, or no-progress), and validates formula-level inference rules. Irreversible actions require Proof for every declared licensing obligation. The checker accepts the two valid certificates and rejects all fifteen unsafe certificates in a semantic battery; four additional structural cases confirm rejection of argument substitution, precondition omission, and risk-flag downgrade. A microbenchmark of a three-obligation payment certificate measures 53.7 μs median and 65.7 μs p95 for construction, checking, serialization, and buffered local persistence. A stateful guardrail with the same certified-state predicate matches PCAA on safety; PCAA’s distinct contribution is a typed, replayable decision artifact. Guarantees are relative to trusted tool contracts: PCAA checks their use, not their real-world semantic correctness.