During the last decade, the cybersecurity literature has conferred a high-level role to machine learning as a powerful security paradigm to recognise malicious software in modern anti-malware systems. However, a non-negligible limitation of machine learning methods used to train decision models is that adversarial attacks can easily fool them. Adversarial attacks are attack samples produced by carefully manipulating the samples at the test time to violate the model integrity by causing detection mistakes. In this paper, we analyse the performance of five realistic target-based adversarial attacks, namely Extend, Full DOS, Shift, FGSM padding + slack and GAMMA, against two machine learning models, namely MalConv and LGBM, learned to recognise Windows Portable Executable (PE) malware files. Specifically, MalConv is a Convolutional Neural Network (CNN) model learned from the raw bytes of Windows PE files. LGBM is a Gradient-Boosted Decision Tree model that is learned from features extracted through the static analysis of Windows PE files. Notably, the attack methods and machine learning models considered in this study are state-of-the-art methods broadly used in the machine learning literature for Windows PE malware detection tasks. In addition, we explore the effect of accounting for adversarial attacks on securing machine learning models through the adversarial training strategy. Therefore, the main contributions of this article are as follows: (1) We extend existing machine learning studies that commonly consider small datasets to explore the evasion ability of state-of-the-art Windows PE attack methods by increasing the size of the evaluation dataset. (2) To the best of our knowledge, we are the first to carry out an exploratory study to explain how the considered adversarial attack methods change Windows PE malware to fool an effective decision model. (3) We explore the performance of the adversarial training strategy as a means to secure effective decision models against adversarial Windows PE malware files generated with the considered attack methods. Hence, the study explains how GAMMA can actually be considered the most effective evasion method for the performed comparative analysis. On the other hand, the study shows that the adversarial training strategy can actually help in recognising adversarial PE malware generated with GAMMA by also explaining how it changes model decisions.

Adversarial machine learning has exposed critical vulnerabilities in Artificial Intelligence-based Windows Portable Executable (PE) malware detection. A well-crafted small perturbation to a PE malware binary can cause it to be misclassified as goodware. Adversarial Training (AT) is one of the most effective defenses; however, it is not always sufficient alone and often suffers from the robustness–accuracy trade-off. This study proposes Adversarial Training with Synthetic Augmentation (ATS), a novel defense methodology that augments unrealistic synthetic adversarial examples into the standard adversarial training, produced using the Conditional Tabular GAN (CTGAN). The robustness and resilience of Random Forest, Light Gradient Boosting Machine (LightGBM), and Multilayer Perceptron (MLP) classifiers were evaluated against five realistic Windows PE adversarial attacks: Full DOS, EXTEND, SHIFT, FGSM-padding, and GAMMA. Results show that the ATS methodology consistently outperformed AT in enhancing robustness across all attacks and classifiers while maintaining or improving clean accuracy and F1-score. SHAP-based interpretability analysis further reveals that ATS reduces dependence on attack-sensitive low-level features and increases attention to stable PE features. Overall, ATS provides a model-agnostic enhancement to standard AT, effectively reducing false negatives without compromising clean accuracy.

Machine Learning (ML)-based malware detectors perform exceptionally well on standard benchmarks, but their robustness to evasive malware remains fragile. On EMBER2024, strong tree-based models achieve high accuracy on clean test sets yet miss a large number of evasive Windows PE samples. Through SHAP-based explainability, matched error comparisons, feature-family analysis, and margin diagnostics on the validation split, we identify consistent attribution and decision-margin differences between baseline true negatives and false negatives. We further show that false negatives occur at different decision depths, including both near-boundary cases and samples misclassified with high benign confidence. We propose C-GUARD, a context-adaptive conformal gated detector that preserves the optimized high-performing baseline while selectively invoking an auxiliary rescue detector on a targeted subset of baseline negative predictions. C-GUARD combines out-of-fold rescue learning to distinguish baseline false negatives from true negatives with adaptive gating under an explicit false-positive budget. A one-sided intervention rule preserves all baseline malware decisions. On the EMBER2024 standard test set, C-GUARD recovers 178 additional malware samples at the cost of 82 additional false positives. On the evasive challenge set, it recovers 17 additional evasive malware samples.

Adversarial evaluation of machine-learning-based network intrusion detection systems (NIDSs) requires an explicit bound on attacker perturbations, yet fixed bounds are often reused across datasets without verifying scale comparability. We evaluate two differentiable detectors on CICIDS2017, TON_IoT, and UNSW-NB15 using a constrained, control-surface-aware evasion procedure in which perturbation cost is measured in interquartile-range units. The measured constrained feature-space evasion cost differs by more than two orders of magnitude across benchmarks. On UNSW-NB15, changing only the perturbation bound from a fixed reference value of 50 to a dataset-calibrated $P_{95}$ value changes measured evasion success from 2.6% to 84.0%, with the conclusion remaining stable across $P_{90}$, $P_{95}$, and $P_{99}$ calibration. We therefore propose a dataset-specific quantile calibration procedure and a monotonicity-based internal validity test exploiting nested attacker action spaces. Calibration removes the severe fixed-bound monotonicity violations observed on the two most poorly scaled datasets, although the test remains diagnostic rather than certifying. A secondary attribution-based robustness estimator does not generalise consistently across the three benchmarks. These results show that perturbation-budget calibration is an essential component of credible cross-dataset adversarial NIDS evaluation.