Despite the recent success of deep-learning models, traditional models are overconfident and poorly calibrated. This poses a serious problem when applied to high-stakes applications. To solve this issue, uncertainty quantification (UQ) models have been developed to allow the detection of misclassifications. Meta-model-based UQ methods are promising due to the lack of predictive model re-training and low resource requirement. However, there are still several issues present in the training process. (1) Most current meta-models are trained using hard labels that do not allow quantification of the uncertainty associated with a given data sample; and (2) in most cases, the base model has a high test accuracy. Therefore, the samples used to train the meta-model primarily consist of correctly classified samples. This leads the meta-model to learn a poor approximation of the true decision boundary. To address these problems, we propose a novel soft-label formulation that better differentiates between correct and incorrect classifications, thereby allowing the meta-model to distinguish between correct and incorrect classifications with high uncertainty (i.e., low confidence). In addition, a novel training framework using adversarial samples is proposed to explore the decision boundary of the base model and mitigate issues related to training datasets with label imbalance. To validate the effectiveness of our approach, we use two predictive models trained on SVHN and CIFAR10 and evaluate performance according to sensitivity, specificity, an F1-score-style metric, average precision, and the Area Under the Receiver Operating Characteristic curve. We find the soft-label approach can significantly increase the model’s sensitivity and specificity, while the training with adversarial samples can noticeably improve the balance between sensitivity and specificity. We also compare our method against four state-of-the-art meta-model-based UQ methods, where we achieve significantly better performance than most models.

Despite the widely reported potential of deep neural networks for automated breast tumor classification and detection, these models are vulnerable to adversarial attacks, which leads to significant performance degradation on different datasets. In this paper, we introduce a novel adversarial attack approach under the decision-based black-box setting, where the attack does not have access to the model parameters, and the returned information from querying the target model consists of only the final class label prediction (i.e., hard-label attack). The proposed attack approach has two major components: adaptive binary search and semantic-aware search. The adaptive binary search utilizes a coarse-to-fine strategy that applies adaptive tolerance values in different searching stages to reduce unnecessary queries. The proposed semantic mask-aware search crops the search space by using breast anatomy, which significantly avoids invalid searches. We validate the proposed approach using a dataset of 3378 breast ultrasound images and compare it with another state-of-the-art method by attacking five deep learning models. The results demonstrate that the proposed approach generates imperceptible adversarial samples at a high success rate (between 99.52% and 100%), and dramatically reduces the average and median queries by 23.96% and 31.79%, respectively, compared with the state-of-the-art approach.

Network security tools are indispensable in testing and evaluating the security of computer networks. Existing tools, such as Hping3, however, offer a limited set of options and attack-specific configurations, which restrict their use solely to well-known attack patterns. Although highly parameterizable libraries, such as Scapy, provide more options and scripting capabilities, they require extensive manual setup and often a steep learning curve. The development of powerful AI models, capitalizing on the transformer architecture, has enabled cybersecurity researchers to develop or incorporate these models into existing cyber-defense systems and red-team assessments. Prominent models such as NetGPT, TrafficFormer, and TrafficGPT can be effective, but require extensive computational resources for fine-tuning and a complex setup to adapt to proprietary networking environments and protocols. In this work, we propose AgentRed, a lightweight tool for generating network attack traffic with minimal human configuration and setup. Our tool integrates an AI agent and a large language model with fewer than a billion parameters into the network traffic generation process. Our method creates lightweight Low-Rank Adaptation (LoRA) adapters that can learn specific traffic patterns in a particular network environment. Our agent can autonomously train the LoRA adapters, search online documentation for attack patterns and parameters, and select appropriate adapters to generate network traffic specific to the user’s needs. It utilizes the LoRA adapters to create an intermediate traffic representation that can be parsed and executed by tools such as Scapy to generate malicious traffic in a virtualized test environment. We assess the performance of the proposed approach on six popular network attacks, including flooding attacks, Smurf, Ping-of-Death, and normal ICMP ping traffic. Our results validate the ability of the proposed tool to efficiently generate network packets with 97.9% accuracy using the LoRA adapters, compared to 95.4% accuracy using the base pre-trained Qwen3 0.6B model. When the AI agent performs online searches to enrich the LoRA adapters’ context during traffic generation, our method maintains an accuracy of 96.0% across all tested traffic patterns.