Network anomaly detection is an effective way for analysing and detecting malicious attacks. However, the typical anomaly detection techniques cannot perform the desired effect in the controlled network just as in the general network. In the circumstance of the controlled network, the detection performance will be lowered due to its special characteristics including the stronger regularity, higher dimensionality and subtler fluctuation of its traffic. On the motivation, the study proposes a novel classifier framework based on cross entropy and support vector machine (SVM). The technique first subtracts the representative traffic characteristics from the network traffic and defines a 7‐tuple feature vector for the controlled network by extending the traditional 5‐tuple representation of the usual network. Then the probability distributions and cross entropies of the 7 tuples are calculated during the defined statistical window so as to generate the 7‐tuple cross‐entropy feature vector for profiling the network traffic fluctuation in the controlled network. Finally, the multi‐class SVM classifier is trained by importing the 7‐tuple cross‐entropy feature vectors. Experimental results show that the proposed classifier can achieve higher detection rates and is more suitable to be used in the controlled network than the typical detection techniques.

While deep learning techniques have been extensively employed in malware detection, there is a notable challenge in effectively embedding malware features. Current neural network methods primarily capture superficial characteristics, lacking in-depth semantic exploration of functions and failing to preserve structural information at the file level. Motivated by the aforementioned challenges, this paper introduces MalHAPGNN, a novel framework for malware detection that leverages a hierarchical attention pooling graph neural network based on enhanced call graphs. Firstly, to ensure semantic richness, a Bidirectional Encoder Representations from Transformers-based (BERT) attribute-enhanced function embedding method is proposed for the extraction of node attributes in the function call graph. Subsequently, this work designs a hierarchical graph neural network that integrates attention mechanisms and pooling operations, complemented by function node sampling and structural learning strategies. This framework delivers a comprehensive profile of malicious code across semantic, syntactic, and structural dimensions. Extensive experiments conducted on the Kaggle and VirusShare datasets have demonstrated that the proposed framework outperforms other graph neural network (GNN)-based malware detection methods.