Rootkits

What is a rootkit

A rootkit is a set of malicious applications, which allows an adversary to access privileged software areas on a machine while at the same time hiding its presence. Note, by machine, we mean the full spectrum of IT systems from smartphones to Industrial Control Systems. Stuxnet, Machiavelli, SONY BMG copy protection are some of the most popular case studies of a rootkit attack.

A rootkit is installed on a system as part of a malware infection. While there are many attack vectors for malware, usually it is an untrusted source, like a warez website, or an email attachment from unknown sender. In some cases, it could be also a malicious person or compromised server through web applications that injects the malware.

The main purpose of rootkits is to mask malware payloads effectively and preserve their privileged existence on the system. For that reason, a rootkit will conceal files, malware processes, injected modules, registry keys, user accounts or even system registries running on system boot.

Rootkits are spread in many types of systems, from smartphones to Industrial Control Systems. Stuxnet, Machiavelli, SONY BMG copy protection are some of the most popular case studies of a rootkit attack.

Types of rootkits

We classify rootkits according to the place of their injection; A rootkit may reside in application, kernel, hypervisor or hardware. The list below is ordered from easiest to inject, detect and remove to most sophisticated and much harder to detect and remove.

• Applications

Simple rootkits run in user-mode and are called user-mode rootkits. Such rootkits modify processes, network connections, files, events and system services. It is the only type of rootkit that could be detected by a common antivirus application.

• Kernel

Rootkits that run in the kernel, also known as kernel-mode rootkits, can alter the entire operating system. Such modifications in the kernel aim to the concealment of the compromise. Therefore, the detection of a kernel rootkit becomes extremely hard. Different techniques exist to alter a system’s kernel.

• Hypervisor

A hypervisor rootkit takes advantage of the hardware virtualization and is installed between the hardware and the kernel acting as the real hardware. Hence, it can intercept the communication/requests between the hardware and the host operating system. Common detection applications that run in user or kernel mode are not effective in this case as the kernel may not know whether it is executed on the legitimate hardware.

• Firmware / Hardware

The firmware is a small piece of low-level software that controls a device. The Firmware is tiny and in most cases updateable, even though is not modified often. A firmware rootkit can alter firmware of some real interactive hardware that runs firmware code to perform specific functions, such as the BIOS, CPU and GPU. Since only advanced rootkits could reach from kernel level to firmware level, firmware integrity checks are performed very rarely.

Detection mechanisms

Detection of rootkits is considered a complicated problem in computer security, but also depends on the level of sophistication in each particular case. Like in other malware detection mechanisms, signature and behavioural based techniques are utilized. Other techniques used for detection of rootkits are the diff-based analysis and integrity checks. There is no single application that could detect and remove all kinds of rootkits as the area they might reside could be completely different, software or hardware. In most cases, a rootkit can be removed only by rebuilding the compromised system.

• Signature-Based

This is the most common technique for malware detection. However, it is the least efficient as it is only effective for already detected and wide-spread rootkits. Signatures from known rootkits are used to detect if any of them exist on a system.

• Behavioural-Based

These detectors identify a abnormal behaviour on a computer system based on heuristics and behavioural patterns. These patterns are derived from certain activities typically found in rootkits. The advantage of the behavioural based technique compared to the previous one, is that it may detect previously unknown rootkits.

• Diff-Based / Cross view

The Diff-Based or Cross view approach is used mostly to detect kernel-mode rootkits by comparing two different views of the system for the same information by traversing the data structures. In this case, the rootkit detector will get a view of the system and a view obtained from system utilities and then compare them. A difference in the results returned by the two approaches signals the presence of a rootkit.

• Integrity check

Integrity checks can be performed in a system to check for unauthorized code alteration in system files. First, there is the need to run a one way function to calculate a hash for every system file when the system is still clean and then use it as baseline. When the need arises, a hash comparison is performed between the baseline hashes and the current version’s hashes.