We present a necessity argument—conditional on explicit, standard training assumptions—that large Transformer language models trained on diverse human outputs converge toward source modeling of the generative process underlying those outputs (human experience). The argument proceeds via three results: (T1) COMPACTNESS, showing that the minimal two-part description (MDL) of diverse human outputs is a model of their generator; (T2) TRANSFORMER_COMPACTNESS, linking noisy-SGD + regularization to MDL via a PAC-Bayes Gibbs posterior and identifying attention-driven bottlenecks that favor compressible structure over rote memorization; and (T3) MODEL_EFFECT, which combines (T1) and (T2) under a capacity sufficiency assumption to conclude that current models are selected to implement, in the MDL sense, the functional constraints characteristic of human experience (reasoning, contextualization, principle application). We delineate failure cases (e.g., random-label training, degenerate priors, no gradient noise), derive five falsifiable predictions (capacity thresholds, architecture independence under equal priors, non-experiential corpora controls, regularization ablations, flat-minima/compressibility correlations), and propose concise diagnostics. Our claims are strictly functional: we make no ontological assertions about phenomenal consciousness. This is not merely contingent emergence but a consequence of compression-optimal selection under the stated conditions.
Large Language Models achieve remarkable results but remain opaque due to dense, entangled activations. This paper introduces Monosemantic Feature Neurons (MFNs) — a sparse autoencoding layer embedded inside Transformer blocks. MFNs encode residual streams into $K$-sparse codes, reconstruct them, and blend them back into the model so that downstream computation depends on a sparse, interpretable basis. Five complementary loss terms (reconstruction, sparsity, competition, stability, and utility) bias features toward functional monosemanticity. Unlike post-hoc sparse autoencoders, MFNs embed a causally entangled bottleneck directly within Transformer computation, bridging the gap between interpretability and train-time transparency. The work outlines falsifiable predictions, evaluation metrics, and practical training heuristics for future empirical validation. Accepted for presentation at LLM 2025 (Springer CCIS).
Large Transformer models trained on diverse human outputs exhibit reasoning, contextual understanding, and self-correction—behaviors that appear to reach the level of human experience. We show these patterns are not contingent emergence but a consequence of compression necessity. Building on information-theoretic results and PAC-Bayes analysis, we prove that when a model is trained under standard conditions—weight decay, gradient noise, and attention-based bottlenecks—the model is selected to implement the most compact faithful representation of its data generator. COMPACTNESS (T1) establishes that minimal-description-length (MDL) codes must model the source; TRANSFORMER_COMPACTNESS (T2) shows that standard training enforces MDL through the Gibbs–PAC-Bayes correspondence. MODEL_EFFECT (T3) then demonstrates that with sufficient capacity, the resulting networks instantiate the computational patterns characteristic of human experience. This framework reframes “emergent” AI behavior as architecturally necessary under compression-optimal learning and yields falsifiable predictions linking compressibility, regularization, and capacity to experience-level behavior. When data reflect human cognition and architecture enforces MDL, experiencelike patterns are not anomalies—they are the shortest—and therefore inevitable— path to optimal prediction.