Malware detection of non-executables has recently been drawing much attention because ordinary users are vulnerable to such malware. Hangul Word Processor (HWP) is software for editing non-executable text files and is widely used in South Korea. New malware for HWP files continues to appear because of the circumstances between South Korea and North Korea. There have been various studies to solve this problem, but most of them are limited because they require a large amount of effort to define features based on expert knowledge. In this study, we designed a convolutional neural network to detect malware within HWP files. Our proposed model takes a raw byte stream as input and predicts whether it contains malicious actions or not. To incorporate highly variable lengths of HWP byte streams, we propose a new padding method and a spatial pyramid average pooling layer. We experimentally demonstrate that our model is not only effective, but also efficient.

The prevalence of non-executable malware is on the rise, presenting a major threat to users, including major public institutions and corporations. While extensive research has been conducted on detecting malware threats, there is a noticeable gap in studying document-type malware compared with executable files. The proposed model will solve this gap by detecting and classifying document-type malware families using script codes, including tags, to write documents and script languages to execute malicious functions. These script codes offer insights into how the malware was constructed and operates on the victim’s system. Additionally, we leverage language models in our approach. Initially, we develop MalCode2Vec to learn associations between source codes and represent them as numeric vectors. Subsequently, we design a Transformer-based model for document malware detection and family classification. Detection is conducted at both the stream and file levels. To solve the class imbalance issue in the malware family, we utilize a generative adversarial network to generate malware samples. Our experimental domain focuses on the Hangul (Korean) word processor, a tool notably used by North Korea in targeting the South Korean government.