Contemporary alignment methods—RLHF, Constitutional AI, Direct Preference Optimization—share an implicit epistemological commitment: that diverse human values can be aggregated into a single utility function or constitution fixed at training time. When principles are elicited from genuinely pluralistic populations, structurally irreducible contradictions emerge; current pipelines resolve them by consensus filters that function, in effect, as majority vote. We argue that this alignment collapse is not an implementation detail but a representational limit: standard uncertainty formalisms (Bayesian, fuzzy, Dempster–Shafer, imprecise probability), in their dominant normalized forms, do not preserve well-supported contradictory evidence without collapsing it into a single ordering. We propose a paraconsistent alternative in which reward signals, policy outputs, and constitutional evaluations are encoded as neutrosophic triplets (T, I, F) ∈ [0, 1]3 with no sum constraint, so that T + F > 1 becomes a representable—and auditable—state of genuine value conflict. We reformulate reward modeling and constitutional aggregation in this framework, introduce a deliberation-threshold decision functional, and report preliminary empirical evidence: across 2,320 evaluations of six frontier model families, models violate the classical sum constraint (T + I + F > 1) on 62–100% of contested baseline prompts and never on tautological controls, while the strict paraconsistent signature (T + F > 1) concentrates selectively on ethical-conflict items (92%) and expands under audit protocols (71%); which conflicts become visible depends strongly on which model adjudicates them (paired Cochran’s Q, p ≈ 10−12). We specify a two-arm training experiment that would confirm or refute the framework’s central operational claim, and close with four testable hypotheses and a research agenda that treats the question aligning with whom? as political and epistemic before it is technical.