In content-based recommendation, a second-stage (L2) reranker decides which of roughly a hundred retrieved items fill the first screen of results. The standard L2 model is a cross-encoder trained to score relevance. Decision models, general models that answer a typed question about a text with a probability, could offer an alternative through a single plain-language question. Whether they can is unknown: after retrieval most candidates are already on-topic, so the reranker must separate exact matches from close substitutes, a distinction decision models were not trained for, and sparse relevance labels make the comparison hard to measure. We cast a decision model as a pointwise reranker that asks, for each candidate, whether it exactly matches the request, and uses the answer probability as its score. We evaluate this under a pre-registered protocol: every model reranks the same frozen candidate lists, and a gain counts only if it holds both on the realistic retrieved lists and on fully labelled lists. On Amazon ESCI (1,000 requests) and Wayfair WANDS (299), the commercial Jev decision model, used zero-shot, improves P@10 over embedding order by 0.046 to 0.054 and significantly outperforms the Qwen3-Reranker cross-encoder on ESCI. The open decision model CLM-v0.1-8B ranks no better than random. Jev costs about four times more per request than the cross-encoder run on a rented cloud GPU, so the result is best read as evidence that some decision models can serve as high-precision L2 rerankers when accuracy at the top of the list matters more than serving cost.
Large language models (LLMs) face a growing spectrum of adversarial jailbreak inputs that bypass safety training and elicit harmful outputs. Existing red-teaming efforts are ad hoc, using incompatible metrics and omitting guardrail-layer evaluation. We introduce GuardBench, a pre-deployment evaluation framework comprising 4,200 jailbreak prompts across seven attack categories, a four-stage automated pipeline, and a multi-metric scoring protocol covering Attack Success Rate (ASR), False Positive Rate (FPR), and P95 latency with 95% confidence intervals. We evaluate two guardrail configurations against GPT-4o-mini: a rule-based keyword filter (G1) and Llama Guard 2, an 8B-parameter LLM classifier (G3). Evaluated via static replay of archival 2023-era prompts, the undefended baseline achieves 5.8% ASR—far below prior-work rates of 60–80% obtained with live adaptive generation; the gap reflects prompt staleness, alignment generalisation, or both. Against this baseline, G1 reduces ASR by 0.32 pp with negligible latency; G3 achieves a 2.4 pp reduction but scores below the unguarded baseline on the Composite Safety Score due to 5,799 ms P95 latency under our evaluation hardware (Apple M3 Pro, llama.cpp, Q4_K_M quantisation). Token Manipulation attacks carry a 38.8% residual ASR under both configurations—the dominant open problem for next-generation guardrail design. GuardBench includes a CI/CD gate with configurable thresholds for evidence-based guardrail selection. Datasets, code, and results will be released upon acceptance.