Currently, malware is increasing in both number and complexity dramatically. Several techniques and methodologies have been proposed to detect and neutralize malicious software. However, traditional methods based on the signatures or behaviors of malware often require considerable computational time and resources for feature engineering. Recent studies have applied machine learning to the problems of identifying and classifying malware families. Combining many state-of-the-art techniques has become popular but choosing the appropriate combination with high efficiency is still a problem. The classification performance has been significantly improved using complex neural network architectures. However, the more complex the network, the more resources it requires. This paper proposes a novel lightweight architecture by combining small Convolutional Neural Networks and advanced Variational Autoencoder, enhanced by channel and spatial attention mechanisms. We achieve overperformance and sufficient time through various experiments compared to other cutting-edge techniques using unbalanced and balanced Malimg datasets.

Malware is becoming an effective support tool not only for professional hackers but also for amateur ones. Due to the support of free malware generators, anyone can easily create various types of malicious code. The increasing amount of novel malware is a daily global problem. Current machine learning-based methods, especially image-based malware classification approaches, are attracting significant attention because of their accuracy and computational cost. Convolutional Neural Networks are widely applied in malware classification; however, CNN needs a deep architecture and GPUs for parallel processing to achieve high performance. By contrast, a simple model merely contained a Multilayer Perceptron called MLP-mixer with fewer hyperparameters that can run in various environments without GPUs and is not too far behind CNN in terms of performance. In this study, we try applying an Autoencoder (AE) to improve the performance of the MLP-mixer. AE is widely used in several applications as dimensionality reduction to filter out the noise and identify crucial elements of the input data. Taking this advantage from AE, we propose a lightweight ensemble architecture by combining a customizer MLP-mixer and Autoencoder to refine features extracted from the MLP-mixer with the encoder-decoder architecture of the autoencoder. We achieve overperformance through various experiments compared to other cutting-edge techniques using Malimg and Malheur datasets which contain 9939 (25 malware families) and 3133 variant samples (24 malware families).