As reverse engineering technology develops rapidly, the financial loss caused by software security issues is urgent. Therefore, how to effectively protect software is a critical problem to solve. The software protection method based on code obfuscation is an effective way, and constructing an effective obfuscation algorithm can increase the cost of reverse software. It is conspicuous that current development of code obfuscation focuses on increasing the complexity of the code structure without paying much attention to the protection of program semantic information, which may help experienced attackers improve their analysis efficiency. This paper proposes COOPS for protecting software based on program semantic information, in which functions are regarded as basic semantic units. The switch relationship between the intrafunction control flow and the interfunction calling is established. The interfunction calling can be hidden in the intrafunction control flow, and in reverse, the intrafunction control flow can also be converted to interfunction calling. In this way, considering intraprogram function semantic unit level discrete, this method reconstructs the intraprogram semantic relationship. To determine the relative effectiveness, we have evaluated COOPS on OpenSSL and SpecInt-2000 test sets. For both of them, the function calling graphs before and after obfuscation differ more than 90, which means COOPS significantly changes the control flow of the program. The evaluation shows that compared with O-LLVM, COOPS manifests strong resistance to Asm2vec and other program similarity analysis techniques and significantly improves the level of software protection rather than necessitating time-consuming and heavyweight problems.

API calls are programming interfaces used by applications. When it is difficult for an analyst to perform a direct reverse analysis of a program, the API provides an important basis for analyzing the behavior and functionality of the program. API address spaces are essential for analysts to identify API call information, and therefore API call obfuscation is used as a protection strategy to prevent analysts from obtaining call information from API address spaces. API call obfuscation avoids direct API calls and aims to create a more complex API calling process. Unfortunately, current API call obfuscation methods are not effective in preventing analysts from obtaining usable information from the API address space. To solve this issue, in this paper, we propose an API call obfuscation model based on address space obscurity. The key functions within the API are encrypted and moved to the user code space for execution. This breaks the relationship between the API and its address space, making it impossible for analysts to obtain address information about a known API from the API address space. In our experiments, we developed an archetypical compiler-level API call obfuscation system to automate the obfuscation of input source code into an obfuscated file. The results show that our approach can thwart existing API deobfuscation techniques and is highly resistant to various open-source dynamic analysis platforms. Compared to other obfuscation techniques, our scheme improves API address space obscurity by more than two times, the detection rate of deobfuscation techniques such as Scylla, etc. is zero, and the increase in obfuscation overhead is not more than 20%. The above results show that APIASO has better obfuscation effect and practicability.

Viruses are one of the main threats to the security of today’s cyberspace. With the continuous development of virus and artificial intelligence technologies in recent years, the intelligentization of virus technology has become a trend. It is of urgent significance to study and combat intelligent viruses. In this paper, we design a new type of confirmatory virus from the attacker’s perspective that can intelligently infect software frameworks. We aim for structural software as the target and use BCSD (binary code similarity detection) to identify the framework. By incorporating a software framework functional structure recognition model in the virus, the virus is enabled to intelligently recognize software framework functions in executable files. This paper evaluates the BCSD model that is suitable for a virus to carry and constructs a lightweight BCSD model with a knowledge distillation technique. This research proposes a software framework functional structure recognition algorithm, which effectively reduces the recognition precision’s dependence on the BCSD model. Finally, this study discusses the next researching direction of intelligent viruses. This paper aims to provide a reference for the research of detection technology for possible intelligent viruses. Consequently, focused and effective defense strategies could be proposed and the technical system of malware detection could be reinforced.