Network protocol vulnerability detection plays an important role in many domains, including protocol security analysis, application security, and network intrusion detection. In this study, by analyzing the general fuzzing method of network protocols, we propose a novel approach that combines network traffic analysis with the binary reverse engineering method. For network traffic analysis, the block-based protocol description language is introduced to construct test scripts, while the binary reverse engineering method employs the genetic algorithm with a fitness function designed to focus on code coverage. This combination leads to a substantial improvement in fuzz testing for network protocols. We build a prototype system and use it to test several real-world network protocol implementations. The experimental results show that the proposed approach detects vulnerabilities more efficiently and effectively than general fuzzing methods such as SPIKE.
Formal techniques have been devoted to analyzing whether network protocol specifications violate security policies; however, these methods cannot detect vulnerabilities in the implementations of the network protocols themselves. Symbolic execution can be used to analyze the paths of the network protocol implementations, but for stateful network protocols, it is difficult to reach the deep states of the protocol. This paper proposes a novel model-guided approach to detect vulnerabilities in network protocol implementations. Our method first abstracts a finite state machine (FSM) model, then utilizes the model to guide the symbolic execution. This approach achieves high coverage of both the code and the protocol states. The proposed method is implemented and applied to test numerous real-world network protocol implementations. The experimental results indicate that the proposed method is more effective than traditional fuzzing methods such as SPIKE at detecting vulnerabilities in the deep states of network protocol implementations.
In this paper a new vulnerability discovery method based on symbolic execution and machine learning is proposed. Given a vulnerable function, similar function set can be located efficiently based on machine learning technique. Function call graphs of each function from the similar function set are obtained by dataflow analysis of the whole program. At last, the symbolic execution engine KLEE is used to travel certain paths of the tested program according to the function call graphs obtained. With the help of the function call graphs, KLEE has no need to travel all of the program paths that can absolutely increase the efficiency of symbolic execution. Upon the dual effect of machine learning and symbolic execution, the method proposed is more efficient than solely using symbolic execution and is more precise than solely using machine learning.
Network protocol binary software is threaten by vulnerabilities. Current analyzing methods do not make full use of the interaction information in network protocol and do not treat both sides of network protocol as a whole system. This paper proposes a novel method based on joint symbolic execution to test synchronously both server and client of the whole network protocol binary software stacks. Moreover, this paper presents a prototype system, S2EProtocol-joint, upon Selective Symbolic Execution (S2E) platform to automatically test network protocol binary software. The experiment results validate that this approach is effective and efficient in detecting vulnerabilities.
Current network protocol binary software testing methods cannot discover serious vulnerabilities in deep states. This article introduces a novel method based on multi-packet symbolic execution, which can drive the software to deep states, to test the whole network protocol binary software stacks. This article also presents a prototype system, S2EProtocol-multi, upon Selective Symbolic Execution (S2E) platform and evaluates it on real-world network protocol binary software. The results validate that the proposed method can explore deep states and detect vulnerabilities efficiently and effectively.