In the late 20th century, computer viruses emerged as powerful malware that resides permanently in target hosts. For a virus to function, it must load into memory from persistent storage, such as a file on a hard drive. Due to the significant destructive potential of viruses, numerous defense measures have been developed to protect computer systems. Among these, antivirus software is one of the most recognized and widely used. Typically, antivirus solutions rely on static analysis (signature-based) technologies to detect infections in files stored on permanent storage devices, such as hard drives or USB (Universal Serial Bus) flash drives. However, a new breed of malware, fileless malware, has been designed to evade detection and enhance durability. Fileless malware resides solely in the memory of the target hosts, circumventing traditional antivirus software, which cannot access or analyze processes executed directly from memory. This study proposes the Check-on-Execution (CoE) kernel-based approach to detect fileless malware on Linux systems. CoE intervenes by suspending code execution before a program executes code from a process’s writable and executable memory area. To prevent the execution of fileless malware, CoE extracts the code from memory, packages it with an ELF (Executable and Linkable Format) header to create an ELF file, and uses VirusTotal for analysis. Experimental results demonstrate that CoE significantly enhances a Linux system’s ability to defend against fileless malware. Additionally, CoE effectively protects against shell code injection attacks, including buffer and memory overflows, and can handle packed malware. However, it is important to note that this study focuses exclusively on fileless malware, and further research is needed to address other types of malware.

This manuscript introduces MPSD (Malicious PowerShell Script Detector), an advanced tool to protect Windows systems from malicious PowerShell commands and scripts commonly used in fileless malware attacks. These scripts are often hidden in Office document macros or downloaded remotely via PowerShell, posing significant threats to corporate networks. A 2018 report revealed that 77% of successful cyberattacks involved fileless malware, with PowerShell being the primary attack method, as highlighted in Red Canary’s 2022 report. To counter these threats, MPSD leverages the Antimalware Scan Interface (AMSI) to intercept and analyze real-time PowerShell scripts, preventing their execution. It further utilizes VirusTotal to filter out malicious scripts. Unlike traditional methods that rely on direct access to scripts, MPSD detects them before execution, addressing the challenge of hidden or obfuscated scripts. Experimental results show that MPSD outperforms well-known antivirus engines, with a low false-negative rate of 1.83%. MPSD is highly effective against evasion techniques like concatenation, encoding, and reordering, making it a robust tool in the cybersecurity landscape.