The exponential growth of encrypted network traffic poses significant challenges for detecting malicious activities online. The scale of emerging malicious traffic is significantly smaller than that of normal traffic, and the imbalanced data distribution poses challenges for detection. However, most existing methods rely on single-category features for classification, which struggle to detect covert malicious traffic behaviors. In this paper, we introduce a novel semi-supervised approach to identify malicious traffic by leveraging multimodal traffic characteristics. By integrating the sequence and topological information inherent in the traffic, we achieve a multifaceted representation of encrypted traffic. We design two independent neural networks to learn the corresponding sequence and topological features from the traffic. This dual-feature extraction enhances the model’s robustness in detecting anomalies within encrypted traffic. The model is trained using a joint strategy that minimizes both the reconstruction error from the autoencoder and the classification loss, allowing it to effectively utilize limited labeled data alongside a large amount of unlabeled data. A confidence-estimation module enhances the classifier’s ability to detect unknown attacks. Finally, our method is evaluated on two benchmark datasets, UNSW-NB15 and CICIDS2017, under various scenarios, including different training set label ratios and the presence of unknown attacks. Our model outperforms other models by 3.49% and 5.69% in F1 score at labeling rates of 1% and 0.1%, respectively.

Third-party library (TPL) reuse may introduce vulnerable or malicious code and expose the software, which exposes them to potential risks. Thus, it is essential to identify third-party dependencies and take immediate corrective action to fix critical vulnerabilities when a damaged reusable component is found or reported. However, most of the existing methods only rely on syntactic features, which results in low recognition accuracy and significantly discounts the detection performance by obfuscation techniques. In addition, a few semantic-based approaches face the efficiency problem. To resolve these problems, we propose and implement a more precise and scalable TPL detection method BBDetector. In addition to syntactic features, we consider the rich function-level semantic features and form a feature vector for each function. Moreover, we design a scalable function vector similarity search method to identify anchor functions and the candidate libraries, based upon which we carry out TPL detection. The experiment results demonstrate that BBDetector outperforms B2SFinder and ModX in terms of effectiveness, efficiency, and obfuscation-resilient capability. For the nix binaries, the F1-score of BBDetector is 1.11% and 11.21% higher than that of ModX and B2SFinder, respectively. Moreover, for the Ubuntu binaries, the F1-score of BBDetector is 1.32% and 14.93% is higher than that of ModX and B2SFinder, respectively. And in terms of efficiency, the detection time of BBDetector is only 30.02% of ModX. Besides, for the obfuscation-resilient capability, BBDetector is much stronger than B2SFinder. BBDetector achieves a F1-score of 71%, slightly lower than the F1-score of 77% achieved with the non-obfuscated binary programs. However, B2SFinder only achieves an F1-score of 28%, much lower than that of 67% achieved with the non-obfuscated binary programs.

The proliferation of taint-style vulnerabilities in embedded devices poses a significant threat to cybersecurity. However, discovering these vulnerabilities is challenging due to their vast number and variety. While current solutions for discovering vulnerabilities in embedded firmware have achieved some success, they suffer from imprecision, are time-consuming, and fail to consider sensitive sinks and constraints. To address these challenges, we propose a novel taint-style vulnerability discovery method called SinkTaint. SinkTaint incorporates backtracking and constraint analysis to achieve high precision and employs a global taint keyword identification strategy to identify implicit taint keywords. It identifies additional sinks using static analysis and performs backtracking analysis to eliminate sanitized sinks, while retrieving the parameter's length for risky sinks. Furthermore, SinkTaint employs dual-label labeling strategies for taint keywords and data, propagating taint labels based on function return values. Finally, SinkTaint employs symbolic execution-based taint analysis to discover taint-style vulnerabilities. We evaluate SinkTaint on datasets released by SaTC and 10 known overflow vulnerabilities. Compared to state-of-the-art methods, including Karonte, SaTC, and EmTaint, SinkTaint demonstrated superior performance, discovering more vulnerabilities with an increase in vulnerability discovery effectiveness by 472%. To date, SinkTaint has identified 21 high-risk taint-style vulnerabilities that were previously undisclosed.

The proliferation of internet traffic encryption has become a double-edged sword. While it significantly enhances user privacy, it also inadvertently shields cyber-attacks from detection, presenting a formidable challenge to cybersecurity. Traditional machine learning and deep learning techniques often fall short in identifying encrypted malicious traffic due to their inability to fully extract and utilize the implicit relational and positional information embedded within data packets. This limitation has led to an unresolved challenge in the cybersecurity community: how to effectively extract valuable insights from the complex patterns of traffic packet transmission. Consequently, this paper introduces the TB-Graph model, an encrypted malicious traffic classification model based on a relational graph attention network. The model is a heterogeneous traffic burst graph that embeds side-channel features, which are unaffected by encryption, into the graph nodes and connects them with three different types of burst edges. Subsequently, we design a relational positional coding that prevents the loss of temporal relationships between the original traffic flows during graph transformation. Ultimately, TB-Graph leverages the powerful graph representation learning capabilities of Relational Graph Attention Network (RGAT) to extract latent behavioral features from the burst graph nodes and edge relationships. Experimental results show that TB-Graph outperforms various state-of-the-art methods in fine-grained encrypted malicious traffic classification tasks on two public datasets, indicating its enhanced capability for identifying encrypted malicious traffic.

As the most crucial link in the network kill chain, exploiting a vulnerability is viewed as one of the most popular attack vectors to get the control authority of the system, which is dangerous for legal users. Therefore, an effective exploit traffic detection method is urgent. However, current methods are almost based on pattern matching, invalid for encrypted traffic. To address this problem, we propose a reverse shell-based exploit traffic detection method, ETDetector. Our key insight is that the reverse shell attack often coexists with vulnerability exploitation as one of the most popular exploit behaviors. So, we first extract the fusion information feature from original features, such as the packet delay sequence, as input of a decision tree model to identify reverse shell traffic in the shellcode execution stage. Then, we trace suspicious traffic in the shellcode delivery stage by reconstructing the session relationship of the two stages above. Compared with Blatta, using a cyclic neural network to detect early exploit traffic, the detection rate of ETDetector is increased by 50% and valid for encrypted exploit traffic. In addition, we propose a traffic stratification method based on a bisecting K-means algorithm, which can intuitively show the traffic communication behavior and improve the interpretability of ETDetector.

Vulnerable open-source component reuse can lead to security problems. At present, open-source component detection for binary programs can only reveal whether open-source components with vulnerabilities are reused, which cannot determine the specific location of vulnerabilities. To address this problem, we propose BMVul, an open-source vulnerability function detection based on the software modularization method, which is oriented to binary programs. BMVul performs binary modularization by the overlapping clustering method DBM based on directed graph, then uses feature comparison technology to carry out modular software component analysis. After creating open-source component vulnerability function set through function signature, BMVul detects vulnerability function in the binary modules reusing open-source components. The experimental results show that compared with the component detection based on Louvain modularization and B2SFinder, BMVul improves the precision by 3.16% and 59.57%, respectively. Moreover, the precision of unique binary module matching is improved by 39.43% compared with the Louvain method. The F1 score is improved by 8.45% compared to B2SFinder. Module-level detection narrows the search space of vulnerability functions, thereby reducing the workload of open-source vulnerability detection, which is of great significance for software security analysis.