The rapid growth of the consumer credit card market has introduced substantial regulatory and risk management challenges. To address these challenges, financial institutions increasingly adopt advanced machine learning models to improve default prediction and portfolio monitoring. However, the use of such models raises additional concerns regarding transparency and fairness for both institutions and regulators. In this study, we investigate the consistency of Shapley Additive Explanations (SHAPs), a widely used Explainable Artificial Intelligence (XAI) technique, through a case study on credit card probability-of-default modeling. Using the Default of Credit Card dataset containing 30,000 consumer credit accounts information, we train 100 Extreme Gradient Boosting (XGBoost) models with different random seeds to quantify the consistency of SHAP-based feature attributions. The results show that the feature SHAP stability is strongly associated with feature importance level. Features with high predictive power tend to yield consistent SHAP rankings (Kendall’s W = 0.93 for the top five features), while features with moderate contributions exhibit greater variability (Kendall’s W = 0.34 for six mid-importance features). Based on these findings, we recommend incorporating SHAP stability analysis into model validation procedures and avoiding the use of unstable features in regulatory or customer-facing explanations. We believe these recommendations can help enhance the reliability and accountability of explainable machine learning framework in credit risk management.

LLM safety classifiers such as Llama Guard detect overtly harmful prompts well but remain vulnerable to jailbreaks that disguise intent via role-play, fictional framing, or indirect requests. We present Reflect-Guard: Llama-Guard-3-8B fine-tuned via QLoRA to generate a distilled, GPT-4o-mini-sourced safety rationale before its verdict. Using only 1000training examples and a five-condition ablation, we isolate reflection’s contribution from that of ordinary supervised fine-tuning (SFT). SFT on verdict labels alone already accounts for most of the aggregate gain over the un-fine-tuned baseline (WildGuardTest F₁: 0.770 → 0.860; JailbreakBench guard bypass rate: 10.3% → 0.0%). Adding reflection does not improve these aggregate numbers further (F₁ falls to 0.842, guard bypass rises to 1.8%) but reallocates performance toward adversarially framed inputs: adversarial-subset recall rises from 0.845 (SFT-only) to 0.921 (+7.6 pp), at a cost of −7.9 pp precision on that subset. The same direction holds across three training seeds (paired t-tests, $p=0.018$ for overall F₁ and $p=0.027$ for adversarial recall), though we interpret these values cautiously given the small number of seeds. Measured inference latency is 17.9× the baseline’s, driven by ∼92 additional reflection tokens per prompt. The same checkpoint, without additional fine-tuning, shows promising zero-shot transfer to a 329-example financial-harm benchmark (F₁: 0.868 → 0.929; recall: 0.767 → 0.867, zero false positives); preliminary two-seed evidence, however, suggests the SFT-only condition may match or exceed this result, so we treat the finance improvement as evidence toward generalization rather than a settled advantage of reflection specifically. Benchmarked against WildGuard, PromptGuard, and a prompted (non-fine-tuned) GPT-4o-mini reasoning classifier on the same sets, WildGuard leads on aggregate WildGuardTest F₁ (0.887), while Reflect-Guard achieves higher adversarial-subset recall than every evaluated classifier, including WildGuard (0.921 vs. 0.827) and the prompted teacher (0.636). Overall, supervised fine-tuning drives most of the aggregate gain over an un-fine-tuned classifier, while the generated safety rationale yields a narrower, latency-costly improvement specifically in adversarial-subset recall, a precision–recall tradeoff relevant to safety-critical deployments that prioritize recall over inference speed.

The increasing development in the consumer credit card market brings substantial regulatory and risk management challenges. The advanced machine learning models applications bring concerns about model transparency and fairness for both financial institutions and regulatory departments. In this study, we evaluate the consistency of one commonly used Explainable AI (XAI) technology, SHAP, for variable explanation in credit card probability of default models via a case study about credit card default prediction. The study shows the consistency is related to the variable importance level and hence provides practical recommendation for credit risk management