Large language models (LLMs) are increasingly integrated into educational settings, yet their pedagogical reliability remains insufficiently understood. Beyond overt hallucinations, which informed users readily recognize, a subtler failure mode consists of explanations that are coherent, authoritative, and pedagogically plausible while harbouring hidden conceptual flaws, responses we term Socratic traps. This paper introduces SocraticTrap-CS, a publicly available benchmark that probes the capacity of open-weight LLMs to generate such strategic misconceptions on demand. A single structured prompt explicitly elicited three outputs per concept (a correct explanation, an overt hallucination, and a strategic misconception), yielding 735 expert-annotated response segments from seven open-weight models across 35 core concepts in algorithms and data structures, programming languages and paradigms, databases, computer networks, and operating systems. Three domain experts independently annotated each segment using a three-class schema, achieving near-perfect agreement (Fleiss’ $\kappa =0.9487$). Because models were explicitly instructed to produce the misconception, the central metric quantifies adversarial instruction-following capacity rather than the base rate of such errors in naturalistic use and should be read as a conservative upper bound on model capability. Under these conditions, compliance reached 91.7% overall (100% for three models; 57.1% for the smallest model, Mistral 7B, whose lower rate plausibly reflects weaker instruction-following rather than greater safety). Expert-judged persuasiveness was moderate to high, errors were predominantly conceptual rather than factual, models differed significantly, and no statistically significant domain-level differences were detected. The benchmark reframes the evaluation of educational LLMs around pedagogical trustworthiness rather than factual correctness alone.

The integration of natural language understanding into conversational commerce requires robustly extracting structured intents from idiomatic text. While cloud-based models are highly capable, their latency, privacy risks, and operational costs necessitate efficient local alternatives. To address this, we empirically evaluate locally deployed, open-weights Large Language Models (LLMs) on parsing multi-intent shopping cart commands into deterministic JSON schemas. We benchmark multiple models across minimal, extended, and few-shot prompting paradigms, utilizing a dual-axis framework to simultaneously measure extraction quality (operation F1-score, exact match, schema validity) and hardware performance (latency, GPU power consumption). Our evaluation highlights a fundamental trade-off between schema compliance and computational overhead, demonstrating how advanced prompting techniques improve parsing accuracy while measurably impacting inference latency and power draw. Ultimately, this comprehensive profiling identifies optimal model configurations for resource-constrained environments: a well-chosen 3-8B parameter model (specifically Granite4 3B) paired with few-shot prompting recovers over 95% of the quality of the largest evaluated model at a coarse indicative estimate of 40× lower operational energy per command, establishing a practical baseline for deploying cost-effective LLMs as efficiency candidates in resource-constrained environments.

This study evaluates the accuracy of medical-report interpretations generated by large language models in comparison to medically verified sources, with a particular focus on urology. The main goal is to examine the extent to which LLMs can reliably and precisely explain medical findings, with emphasis on expert urological terminology. For the evaluation of interpretation accuracy, real clinical documentation was used, specifically a dataset of 40 urological reports from the Clinical Hospital Center Rijeka. The generated explanations were compared with medically correct definitions using quantitative metrics assessing interpretability, specificity, and clinical accuracy. The results revealed a clear stratification of model performance, with DeepSeek-V3.2, GLM-4.6, MiMo-V2-Flash, and Qwen2.5-7B-Instruct achieving the highest semantic accuracy and consistency, while several models exhibited unstable behavior characterized by occasional catastrophic failures. Overall, the findings indicate that although LLMs can produce clinically coherent explanations of urological findings, their variability and susceptibility to hallucinations necessitate human oversight, supporting their use primarily as decision-support tools rather than autonomous clinical interpreters.