The automated detection of hate speech is a critical but difficult task due to its subjective, behavior-driven nature, which leads to frequent annotator disagreement. While advanced models (e.g., transformers) are state-of-the-art, it is unclear how their performance is affected by the methodological choice of label aggregation (e.g., majority vote vs. unanimous agreement) and task complexity. We conduct a 2x2 quasi-experimental study to measure the compounding impact of these two factors: Labeling Strategy (low-ambiguity ``Pure'' data vs. high-ambiguity ``Majority'' data) and Task Granularity (Binary vs. Multi-class). We evaluate five models (Logistic Regression, Random Forest, Light Gradient Boosting Machine [LightGBM], Gated Recurrent Unit [GRU], and A Lite BERT [ALBERT]) across four quadrants derived from the HateXplain dataset. We find that (1) ALBERT is the top-performing model in all conditions, achieving its peak F1-Score (0.8165) on the ``Pure'' multi-class task. (2) Label ambiguity is strongly associated with performance loss; ALBERT's F1-Score drops by $\approx$15.6\% (from 0.8165 to 0.6894) when trained on the higher-disagreement ``Majority'' data in the multi-class setting. (3) This negative effect is compounded by task complexity, with the performance drop being nearly twice as severe for the multi-class task as for the binary task. A sensitivity analysis confirmed this drop is not an artifact of sample size. We conclude that in HateXplain, behavioral label ambiguity is a more significant bottleneck to model performance than model architecture, providing strong evidence for a data-centric approach.

Spam on messaging apps like Telegram is a serious threat to user security andexperience. In this paper, we compared several machine learning (ML) and deep learning (DL)models to find the most effective way to detect it. We tested our models on a dataset of 20,348messages. We put classic approaches like Logistic Regression and Tree-based Modelsincluding bagging and boosting against modern neural networks—a GRU and the ALBERTtransformer. The results demonstrate that both GRU and ALBERT were the clear winners. TheALBERT model was the top performer, achieving state-of-the-art results with a weightedF1-score of 0.97 and an AUC of 0.9943. The GRU model also delivered excellent performance,with an F1-score of 0.94. Their real strength was in identifying the tricky minority ‘spam’class. Here, ALBERT reached an F1-score of 0.95, and the GRU model scored 0.90,significantly outperforming the other methods. We used McNemar's test to confirm thesefindings were statistically significant. Ultimately, our study sets a new benchmark for spamdetection. It proves that transformer models can effectively secure messaging platforms usingonly the content of the message itself.

Introduction: Online cancer support forums contain naturalistic accounts of fear, uncertainty, coping, and caregiver strain. Such text may contribute to digital phenotyping as one component of longitudinal, human-supervised monitoring, but the operational link between psychosocial distress and sentiment labels requires explicit evaluation. Methods: We benchmarked representative classical, recurrent, and transformer-based models for four-class ordinal sentiment classification using the Mental Health Insights—Vulnerable Cancer Patients dataset ( N = 10,392). Models were evaluated under a single validation-guided 60/20/20 holdout split using weighted F1, macro one-vs-rest AUC, class-specific performance, and paired comparisons. Results: Transformer models achieved the strongest overall performance. ALBERT produced the highest weighted F1 and macro AUC (0.7667 and 0.931, respectively), while BioBERT was closely comparable (weighted F1 = 0.7613; macro AUC = 0.917) and showed slightly higher recall for the “very negative” class (0.8019 vs. 0.7736). Error analysis showed that transformer errors concentrated around ordinal decision boundaries, while residual positive-class errors remained operationally important for supportive workflows. Discussion: These split-specific findings support transformer fine-tuning as a decision-support component for vulnerability-oriented monitoring, while emphasizing calibration, transparent error review, and human oversight rather than autonomous clinical assessment.

Automated content moderation is a critical AI security task. However, models often fail in the nuanced, subjective task of distinguishing “hate” from “offensive” speech. The influential ‘HateXplain‘ benchmark attributed this poor performance to a lack of model explainability, proposing rationale-based training as a solution. In this paper, we challenge this premise. We hypothesize that the models’ unreliability stems from a more fundamental, unaddressed security flaw: a crisis of data integrity caused by high label ambiguity. The original dataset relies on a “majority vote” to assign groundtruth labels, which masks significant annotator disagreement and introduces noise. To test our hypothesis, we isolate this variable. We partition the ‘HateX- plain‘ dataset into two cohorts: (1) a “noisy” Majority-Label set (using standard 2-1 majority votes) and (2) a “clean” Pure-Label set (using only 3-0 unanimous-consensus votes). We then rigorously benchmark five models (Logistic Regression, Random Forest, LightGBM, GRU, and AL- BERT) on both datasets. Our results are conclusive. All models trained on the “Pure-Label” data achieved statistically significant and substantially higher performance. The ALBERT model’s weighted F1-score, for instance, rose from 0.7447 on the “noisy” data to 0.8126 on the “clean” data. This demonstrates that label ambiguity is a more dominant performance bottleneck than the architectural factors previously considered. We conclude that for building secure and reliable AI safety systems, addressing foundational data integrity and label consensus is a more critical challenge than model-level explainability.

The automated detection of hate speech is a critical but difficult task due to its subjective, behavior driven nature, which leads to frequent annotator disagreement. While advanced models (e.g., transformers) are state-of-the-art, it is unclear how their performance is affected by the methodological choice of label aggregation (e.g., ‘majority vote’ vs. ‘unanimous agreement’) and task complexity. We conduct a 2x2 quasi-experimental study to measure the compounding impact of these two factors: Labeling Strategy (low-ambiguity ‘Pure’ data vs. high-ambiguity ‘Majority’ data) and Task Granularity (Binary vs. Multi-class). We evaluate five models (Logistic Regression, Random Forest, LightGBM, GRU, and ALBERT) across four quadrants derived from the HateXplain dataset. We find that (1) ALBERTisthetop-performing modelinall conditions, achieving its peak F1-Score (0.8165) on the ‘Pure’ multi-class task. (2) Label ambiguity is the primary driver of performance loss; ALBERT’s F1-Score drops by ≈15.6% (from 0.8165 to 0.6894) when trained on noisy ‘Majority’ data in the multi-class setting. (3) This negative effect is compounded by task complexity, with the performance drop being nearly twice as severe for the multi-class task as for the binary task. A sensitivity analysis confirmed this drop is attributable to data quality (noise), not sample size. We conclude that behavioral label ambiguity is a more significant bottleneck to mode performance than model architecture, providing strong evidence for a data-centric approach.