What is rootkit malware?

A rootkit is a particularly nasty piece of malware that doesn’t behave like your typical virus. Rootkits insert themselves into the very heart of the operating system; usually at or below the kernel level. This makes them extremely difficult to detect and sometimes impossible to remove. Specific antivirus programs specialize in the detection and removal of rootkits. Below we list the five best anti-rootkit programs.

Some background on why rootkits are so evil

In the run of a day you probably use many different programs on your computer. Different classes of programs need different permissions in order to do their job. The operating system heart, the kernel, needs to have absolute control over every piece of hardware and software in the computer in order to do its job. On the other hand, applications that us humans directly interact with, such as word processors and web browsers, need relatively little control to do their job. Conceptually, these different levels of control are illustrated in the protection ring model with the all-powerful kernel inhabiting Ring Zero and mere human applications in the outer rings. Rootkits typically install themselves into Ring Zero and thus inherit the highest level of access possible.

Rootkits are so named because the first rootkits targeted Unix-like operating systems. The most privileged user on these systems is named root, ergo a rootkit is an application that provides root access to the system. The name stuck regardless of operating system and today even Windows rootkits bear that name despite having no such root user on the system.

While there are examples of beneficial, or at least benign, rootkits, they are generally considered to be malicious. Once installed, a rootkit has the ability to alter virtually every aspect of the operating system and to also completely hide its existence from most antivirus programs. Kernel rootkits are extremely hard to detect and sometimes the only way to ensure the computer is clean is to fully reinstall the operating system. Re-installation will still not help against the even more nefarious firmware rootkits that can live in a system BIOS and survive operating system reinstalls.

Rootkit types

Kernel rootkits

Kernel rootkits operate at Ring Zero and are injected into the kernel. In practice, that means kernel modules for Linux, macOS and other Unix-like operating systems, and Dynamic Link Libraries (DLLs) for Windows systems. They operate at the same level and security posture as the kernel itself, which makes them almost impossible to detect or remove if detected.

User space rootkits

The parts of the operating system that are accessed by the programs you use during your day is collectively referred to as user space or user land. Those terms simply mean that those memory and file areas are unprivileged and applications can access those things without having a high level of permissions.

By definition, rootkits that operate in user space do not have kernel access so they are at a disadvantage in avoiding detection. User space rootkits are usually targeted at specific applications. When that application runs, the rootkit patches the legitimate application in user space memory and hijacks its operation. This type of rootkit is easier to deploy, but is also easier to detect and more prone to giving itself up by causing system crashes.

Bootkits

These are rootkits that are bootable. Your computer’s operating system is bootable, otherwise the computer would not be able to start up. A typical rootkit loads itself during the operating system boot sequence. A bootkit doesn’t need an operating system to do that for it because the bootkit can boot all by itself, and then load the operating system afterwards.

A common aim of bootkits is to subvert things like digital signature verification on kernel modules. This gives the attacker the ability to stealthily load modified modules and files during the boot process, providing access to the machine.

Firmware rootkits

Firmware is the term for something that lies in between hardware and software. Hardware is something that needs to be physically bolted into a computer, whereas software is just code that is introduced into the computer, such as a word processor. Firmware is hardware, usually a chip of some sort, which has the ability to have software loaded into it. Unlike normal software installation that just adds code to the computer, updating firmware software generally involves replacing the entire code base on the chip in one fell swoop with a process known as flashing.

This type of rootkit is normally seen in computer BIOSes or purpose-specific devices such as routers and mobile phones. Because the rootkit lives in firmware, formatting the computer’s hard drive and reinstalling the operating system will have no effect and will not remove the rootkit.

Where do rootkits come from?

Rootkits are usually installed by malicious attackers through the same common vectors as any malware. Phishing remains a very successful way to trick users into installing rootkits. Even though users will be prompted to authorize the installation of the rootkit, many of us have become numb to these constant prompts and will allow it.

In rarer cases, a reputable company may include a rootkit in its own software. In a widely publicised series of terrible decisions in 2005, Sony BMG included a rootkit in its CDs to prevent copying. That led to losing a multi-million dollar class action lawsuit due to the inherent insecurities that the rootkit contained above and beyond its intended purpose as a Digital Rights Management (DRM) tool.

5 free rootkit removal, detection and scanner programs

There are some anti-rookit programs that target a specific rootkit such as Kaspersky’s TDSSKiller, but we’ll deal with more general rootkit detectors. If you are in the unenviable position of already being infected with an identified rootkit, you may wish to search to see if an antivirus vendor has a specific tool for that rootkit.

chkrootkit (Check Rootkit)

Pros: Can be run post-infection Cons: No Windows support. Supported OSes: Linux, FreeBSD, OpenBSD, NetBSD, Solaris, HP-UX, Tru64, BSDI, and macOS

“Check Rootkit” (chkrootkit) is an open source rootkit detector that has been around for a long time. The current version as of this article was released in May of 2017 and can detect 69 different rootkits.

You’ll need a seasoned systems administrator to decipher chkrootkit’s output. Also, true to its name, chkrootkit only checks for rootkits; it can’t remove them. It examines your system files for common signs of rootkits such as:

Recently deleted log files

Log files are great tools for analyzing what has happened to a system. However, since a rootkit has the ability to modify any system file that means it has the ability to modify log file contents or delete logs altogether. chkrootkit tries to detect if the various important log files that record logins such as wtmp and utmp have been altered or recently cleared altogether.

State of network interfaces

TCP/IP networking essentially passes packets around the internet. At every stage of the journey, each packet is addressed to either an internet protocol (IP) address, or a local media access control (MAC) address. Routers on the internet or other networks use a packet’s destination IP address to get it to the proper network. Once the packet arrives in the destination network, the MAC address is used for the final delivery to the proper network card, or network interface controller (NIC).

During normal operation, a NIC will only accept packets addressed to its own MAC address, or broadcast traffic, and it will discard any other packets. It’s possible to put a network interface into promiscuous mode which means the network interface will accept all packets regardless of what NIC the packet is addressed to.

Promiscuous mode is typically only used in network analysis to perform packet sniffing or other types of traffic inspection. It would be unusual for a NIC to operate that way during day-to-day operation. chkrootkit will detect if any of the network cards on the system are operating in promiscuous mode.

Loadable kernel module trojans (LKM trojans)

As covered earlier in this article, the most difficult type of rootkits to detect and clean are kernel module rootkits. They operate at the lowest level of the computer in Ring Zero. These rootkits have the same high level of permissions as the operating system kernel itself. chkrootkit has some ability to detect this type of rootkit.

rkhunter (Rootkit Hunter)

Pros: Mature product Cons: Has to be installed pre-infection Supported OSes: Unix-like operating system such as Linux

From the rkhunter README: “Rootkit Hunter is a host-based, passive, post-incident, path-based tool.” That’s a mouthful, but it tells us a lot.

It’s host based meaning that it is designed to scan the host it is installed on, rather than remote hosts elsewhere on the network.

Post-incident means that it does nothing to harden the system against a rootkit infection. It can only detect if an attack has happened or is in progress.

rkhunter primarily detects rootkits by looking for unrecognized changes in significant files. Before it can recognize changes, it has to know what all those files should look like when they’re clean. It’s therefore critical that rkhunter be installed onto a clean system so it can determine a clean baseline to use for subsequent scans. Running rkhunter on an already infected system will be of limited use since it will not have a complete view of what the clean system should look like.

Most antivirus programs use heuristics to some extent, which means that they look for things that look like viruses, even if it doesn’t specifically recognize every virus. rkhunter has no ability to look for rootkit-like things; it is path-based meaning it can only look for rootkits it already knows about.

OSSEC

Pros: Mature software with a large user base. Can be used post-infection Cons: Aimed at advanced users; complete host intrusion detection system rather than just a rootkit scanner Supported OSes: Linux, BSD, Solaris, macOS, AIX (agent), HP-UX (agent), Windows XP, 2003 server, Vista, 2008 server, 2012 server (agent)

OSSEC is a Host Intrusion Detection System (HIDS) that was founded as an Open Source project. It was acquired by Third Brigade, Inc. which was in turn acquired by Trend Micro. Trend is the current owner and OSSEC remains Free/Libre Open Source Software (FLOSS).

The basic architecture is an OSSEC manager installed on a Unix-like central server that then talks to remote agents on the target systems. It is this agent architecture that allows OSSEC to support such a wide range of operating systems. In addition, some devices such as routers and firewalls can be used agentless meaning no software needs to be installed on them because they inherently possess the ability to talk directly to the OSSEC manager.

OSSEC’s rootkit detection is a mix of file-based analysis and other tests across the entire system. Some of the things OSSEC checks are:

  • network interfaces in promiscuous mode that are not reported as such by other tools like netstat.
  • ports that are not reported in use, but OSSEC is unable to bind to.
  • comparing the output of pid-identifying tools with the output of system level tools like ps.
  • detection of suspicious or hidden files.

GMER

Pros: Can remove some rootkits instead of just detection. Can be used post-infection. Cons: Windows only Supported OSes: Windows XP/VISTA/7/8/10

GMER is a rootkit detector and remover that run on Windows XP/VISTA/7/8/10. It has been around since 2006 and the current version supports 64-bit Windows 10. It was created by a programmer named Przemysław Gmerek, which gives us a hint as to the origin of its name.

Unlike chkrootkit and rkhunter, GMER can not only detect rootkits, but also remove some of them. There’s a version of GMER integrated with the Avast! antivirus software that provides pretty good all-around protection for both viruses and rootkit infections.

GMER doesn’t have to have any special knowledge of the system it is scanning. This means that it can be a post-event scan and detect rootkits even if it was not on the system prior to the rootkit infection.

Rather than comparing files or paths to detect rootkits, GMER concentrates on Windows-centric artifacts such as hidden processes, hidden services, and modified modules. It also looks for hooks which are malicious applications that attach themselves to legitimate processes in order to hide their existence.

Open Source Tripwire

Cons: Needs to be installed and initialized pre-infection Pros: Mature product with a large user base Supported OSes: Linux-based systems

Open Source Tripwire is a host based intrusion detection system (HIDS). The contrast here is compared to a network intrusion detection system (NIDS). Tripwire scans a local computer’s file system and compares its files to a known, good set of files.

Much like rkhunter, Tripwire must be installed onto a clean system prior to any possible infection. It then scans the file system and creates hashes or other identifying information about the files on that system. Subsequent Tripwire scans are then able to pick up changes to those files and alert the systems administrator of those changes.

There are two versions of Tripwire; the commercial products from Tripwire, Inc. and the Open Source version that was originally provided by Tripwire, Inc. in 2000. The commercial version offers a much broader array of products including hardening, reporting, and support for non-Linux operating systems.

While Tripwire isn’t a rootkit detector per se, it can detect rootkit activity that affects and changed files on the system. It doesn’t have any ability to remove rootkits, or to even say with certainty whether a rootkit exists. A skilled administrator will have to interpret the scan results to determine if any action needs to be taken.

Protecting your systems

Keep in mind that a rootkit is malware. It’s really bad malware, but it’s still just malware. The best practices which will protect your system from any type of virus will go a long way to protecting your systems against rootkits as well:

  • Ensure users have the least amount of permissions they need to do their job
  • Educate users how to avoid becoming phishing victims
  • Consider disabling USB and CD drives to avoid people bringing malware from home
  • Ensure antivirus is running on all systems and is up to date
  • Use a firewall to stop unsolicited traffic from entering or exiting your system

In addition to those general steps, rootkit protection requires a proactive stance. Install a rootkit detector now, initialize it, and run it at least daily if not more often. While it’s true that if a system becomes infected with a rootkit that system is probably garbage, the more nefarious situation is that a rootkit lives on your systems for months or years without you knowing. Rootkits can silently ship your precious data off-site without any clue it’s happening until you read about it in the morning paper. If you’ve deployed a rootkit detector you stand a good chance of being alerted this is happening as soon as practical.

Privilege Ring Hertzsprung at English Wikipedia