With the growing diversity of cyberattacks in recent years, anomaly-based intrusion detection systems that can detect unknown attacks have attracted significant attention. Furthermore, a wide range of studies on anomaly detection using machine learning and deep learning methods have been conducted. However, many machine learning and deep learning-based methods require significant effort to design the detection feature values, extract the feature values from network packets, and acquire the labeled data used for model training. To solve the aforementioned problems, this paper proposes a new model called DOC-IDS, which is an intrusion detection system based on Perera’s deep one-class classification. The DOC-IDS, which comprises a pair of one-dimensional convolutional neural networks and an autoencoder, uses three different loss functions for training. Although, in general, only regular traffic from the computer network subject to detection is used for anomaly detection training, the DOC-IDS also uses multi-class labeled traffic from open datasets for feature extraction. Therefore, by streamlining the classification task on multi-class labeled traffic, we can obtain a feature representation with highly enhanced data discrimination abilities. Simultaneously, we perform variance minimization in the feature space, even on regular traffic, to further improve the model’s ability to discriminate between normal and abnormal traffic. The DOC-IDS is a single deep learning model that can automatically perform feature extraction and anomaly detection. This paper also reports experiments for evaluating the anomaly detection performance of the DOC-IDS. The results suggest that the DOC-IDS offers higher anomaly detection performance while reducing the load resulting from the design and extraction of feature values.

Operating systems adopt kernel protection methods (e.g., mandatory access control, kernel address space layout randomization, control flow integrity, and kernel page table isolation) as essential countermeasures to reduce the likelihood of kernel vulnerability attacks. However, kernel memory corruption can still occur via the execution of malicious kernel code at the kernel layer. This is because the vulnerable kernel code and the attack target kernel code or kernel data are located in the same kernel address space. To gain complete control of a host, adversaries focus on kernel code invocations, such as function pointers that rely on the starting points of the kernel protection methods. To mitigate such subversion attacks, this paper presents multiple kernel memory (MKM), which employs an alternative design for kernel address space separation. The MKM mechanism focuses on the isolation granularity of the kernel address space during each execution of the kernel code. MKM provides two kernel address spaces, namely, i) the trampoline kernel address space, which acts as the gateway feature between user and kernel modes and ii) the security kernel address space, which utilizes the localization of the kernel protection methods (i.e., kernel observation). Additionally, MKM achieves the encapsulation of the vulnerable kernel code to prevent access to the kernel code invocations of the separated kernel address space. The evaluation results demonstrated that MKM can protect the kernel code and kernel data from a proof-of-concept kernel vulnerability that could lead to kernel memory corruption. In addition, the performance results of MKM indicate that the system call overhead latency ranges from 0.020 μs to 0.5445 μs, while the web application benchmark ranges from 196.27 μs to 6, 685.73 μs for each download access of 100,000 Hypertext Transfer Protocol sessions. MKM attained a 97.65% system benchmark score and a 99.76% kernel compilation time.

Memory corruption can modify the kernel data of an operating system kernel through exploiting kernel vulnerabilities that allow privilege escalation and defeats security mechanisms. To prevent memory corruption, the several security mechanisms are proposed. Kernel address space layout randomization randomizes the virtual address layout of the kernel. The kernel control flow integrity verifies the order of invoking kernel codes. The additional kernel observer focuses on the unintended privilege modifications. However, illegal writing of kernel data is not prevented by these existing security mechanisms. Therefore, an adversary can achieve the privilege escalation and the defeat of security mechanisms. This study proposes a kernel data protection mechanism (KDPM), which is a novel security design that restricts the writing of specific kernel data. The KDPM adopts a memory protection key (MPK) to control the write restriction of kernel data. The KDPM with the MPK ensures that the writing of privileged information for user processes and the writing of kernel data related to the mandatory access control. These are dynamically restricted during the invocation of specific system calls and the execution of specific kernel codes. Further, the KDPM is implemented on the latest Linux with an MPK emulator. The evaluation results indicate the possibility of preventing the illegal writing of kernel data. The KDPM showed an acceptable performance cost, measured by the overhead, which was from 2.96% to 9.01% of system call invocations, whereas the performance load on the MPK operations was 22.1ns to 1347.9ns. Additionally, the KDPM requires 137 to 176 instructions for its implementations.