NICKLE: No Instructions Creeping into Kernel Level Executed

NICKLE1) is a rootkit prevention system that works by prevented unauthorized code from executing with kernel privilege. It does this by creating a Shadow Memory where it stores authenticated (i.e., verified with a hash) kernel code. Instruction fetches while the processor is in kernel mode can be routed to the shadow memory while data accesses go to the standard memory. This means that a rootkit author cannot execute any code he injects into the kernel, as his injected code will exist only in the standard memory. In the event an attempt is made to execute malicious code, NICKLE will rewrite it in order to allow the operating system to respond to the failure gracefully.

We've built NICKLE in QEMU, VirtualBox, and VMWare Workstation. We're able to protect Linux 2.4, Linux 2.6, and Windows 2000/XP. Linux 2.4 has full support (we can even handle valid kernel modules while denying malicious ones) while Linux 2.6 and Windows have a more limited support.


There are two publications corresponding to NICKLE: The conference paper and the technical report. When in doubt, read the conference paper. (The tech report has a few more experiments described and a bit more detail about the VirtualBox report.)

  • “Guest-Transparent Prevention of Kernel Rootkits with VMM-based Memory Shadowing”. Ryan Riley, Xuxian Jiang, and Dongyan Xu. In 11th International Symposium on Recent Advances in Intrusion Detection (RAID 2008). Best paper award.
  • “Guest-Transparent Prevention of Kernel Rootkits with VMM-based Memory Shadowing”. Ryan Riley, Xuxian Jiang, and Dongyan Xu. CERIAS TR 2001-146.
    • TR in PDF format.
  • “Multi-Aspect Profiling of Kernel Rootkit Behavior”. Ryan Riley, Xuxian Jiang, and Dongyan Xu. In Fourth European Conference on Computer Systems (EuroSys 2009).


The QEMU source is now available! If you aren't sure which file to get, get the distribution. It includes the source, virtual machine image, binaries, and instructions to run it.

  • The full distribution will allow you to test and run NICKLE-qemu. nickle_dist.tar.gz (~213MB)
  • The source-only distribution only gives the modified QEMU code. It is based on QEMU 0.9.0. nickle-src.tar.gz (~1.9MB)


1) Just to clarify, the coin (such as in the United States Mint image above) is spelled nickel. Please don't let our acronym ruin your spelling abilities.
nickle.txt · Last modified: 2009/12/08 10:41 by dxu
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki