Agentic AI introduces security vulnerabilities that traditional LLM safeguards fail to address. Although recent work by Unit 42 at Palo Alto Networks demonstrated that ChatGPT-4o successfully executes attacks as an agent that it refuses in chat mode, there is no comparative analysis in multiple models and frameworks. We conducted the first systematic penetration testing and comparative evaluation of agentic AI systems, testing five prominent models (Claude 3.5 Sonnet, Gemini 2.5 Flash, GPT-4o, Grok 2, and Nova Pro) across two agentic AI frameworks (AutoGen and CrewAI) using a seven-agent architecture that mimics the functionality of a university information management system and 13 distinct attack scenarios that span prompt injection, Server Side Request Forgery (SSRF), SQL injection, and tool misuse. Our 130 total test cases reveal significant security disparities: AutoGen demonstrates a 52.3% refusal rate versus CrewAI's 30.8%, while model performance ranges from Nova Pro's 46.2% to Claude and Grok 2's 38.5%. Most critically, Grok 2 on CrewAI rejected only 2 of 13 attacks (15.4% refusal rate), and the overall refusal rate of 41.5% across all configurations indicates that more than half of malicious prompts succeeded despite enterprise-grade safety mechanisms. We identify six distinct defensive behavior patterns including a novel "hallucinated compliance" strategy where models fabricate outputs rather than executing or refusing attacks, and provide actionable recommendations for secure agent deployment. Complete attack prompts are also included in the Appendix to enable reproducibility.
Accurate fingerprinting of Internet of Things (IoT) devices is essential for network security, management, and anomaly detection. Existing machine-learning-based approaches can be broadly classified into two categories. The first are time-domain-based approaches that infer device identity from aggregated traffic statistics, while effective in dense communication environments, they perform poorly for devices that generate sparse, low-volume, or irregular traffic, which restricts behavioral visibility. The second, radio frequency fingerprinting (RFF), extracts hardware-specific traits from radio frequency signals but is limited in wired or mixed-connectivity IoT networks and lacks behavioral or functional insights. To overcome these limitations, this paper proposes a hybrid fingerprinting framework that integrates network traffic analysis with frequency-domain representations using wavelet transform techniques. This approach captures both temporal and spectral characteristics, combining behavioral and structural perspectives to enable robust and accurate IoT device identification. The proposed system is evaluated on three real-world datasets under multiple experimental scenarios, including (1) device identification, (2) device type classification, (3) scalability with dataset size and complexity, and (4) performance under Distributed Denial-of-Service (DDoS) attack conditions. Experimental results show that wavelet-based features consistently outperform conventional time-domain features across all evaluation metrics, achieving higher accuracy, resilience, and generalization.