initial commit

Makefile

@@ -0,0 +1,41 @@
+# sand sifter make file
+#
+# in x86, instructions run in 32 bit mode sometimes differ from the same
+# instructions run in 64 bit mode.  for this reason, it can be beneficial to
+# fuzz both 32 and 64 bit instructions.  this requires a 32 and 64 bit binary.
+# afaict, capstone will not let you simultaneously install both 32 and 64 bit
+# versions.  to overcome this, we statically link to capstone.  to build both a
+# 32 bit and 64 bit injector:
+#
+# - build and install 32 bit capstone:
+#   ./make.sh nix32
+#   sudo ./make.sh nix32 install
+#
+# - build the 32 bit injector:
+#   make CFLAGS=-m32
+#   mv injector injector_32
+#
+# - build and install 64 bit capstone:
+#   ./make.sh
+#   sudo ./make.sh install
+#
+# - build the 64 bit injector:
+#   make injector
+#   mv injector injector_64
+#
+# you can now copy injector_32 and injector_64 to 'injector' before running
+# ./sifter.py in order to explore that facet of the architecture.
+#
+#TODO: i don't know if i was ever able to get a statically linked capstone to
+# work like i describe above
+
+all: injector
+
+injector: injector.o
+	$(CC) $(CFLAGS) $< -O3 -Wall -l:libcapstone.a -o $@ -pthread
+
+%.o: %.c
+	$(CC) $(CFLAGS) -c $< -o $@ -Wall
+
+clean:
+	rm *.o injector

README.md

@@ -0,0 +1,238 @@
+## s a n d s i f t e r 
+: the x86 processor fuzzer
+
+### Overview
+
+The sandsifter audits x86 processors for hidden instructions and hardware bugs,
+by systematically generating machine code to search through a processor's
+instruction set, and monitoring execution for anomalies. Sandsifter has
+uncovered secret processor instructions from every major vendor; ubiquitous
+software bugs in disassemblers, assemblers, and emulators; flaws in enterprise
+hypervisors; and both benign and security-critical hardware bugs in x86 chips.
+
+With the multitude of x86 processors in existence, the goal of the tool is to
+enable users to check their own systems for hidden instructions and bugs.
+
+To run a basic audit against your processor:
+
+```
+sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
+```
+
+![demo_sandsifter](references/sandsifter.gif)
+
+The computer is systematically scanned for anomalous instructions.  In the upper
+half, you can view the instructions that the sandsifter is currently testing on
+the processor.  In the bottom half, the sandsifter reports anomalies it finds.
+
+The search will take from a few hours to a few days, depending on the speed of
+and complexity of your processor.  When it is complete, summarize the results:
+
+```
+./summarize.py data/log
+```
+
+![demo_summarizer](references/summarizer.png)
+
+Typically, several million undocumented instructions on your processor will be
+found, but these generally fall into a small number of different groups.  After
+binning the anomalies, the summarize tool attempts to assign each instruction to
+an issue category:
+
+* Software bug (for example, a bug in your hypervisor or disassembler),
+* Hardware bug (a bug in your CPU), or
+* Undocumented instruction (an instruction that exists in the processor, but is
+  not acknowledged by the manufacturer)
+
+Press 'Q' to quit and obtain a text based summary of the system scan:
+
+The results of a scan can sometimes be difficult for the tools to automatically
+classify, and may require manual analysis. For help analyzing your results, feel
+free to send the ./data/log file to [email protected].  No personal
+information, other than the processor make, model, and revision (from
+/proc/cpuinfo) are included in this log.
+
+
+### Results
+
+Scanning with the sandsifter has uncovered undocumented processor features
+across dozens of opcode categories, flaws in enterprise hypervisors, bugs in
+nearly every major disassembly and emulation tool, and critical hardware bugs
+opening security vulnerabilities in the processor itself.
+
+Details of the results can be found in the project 
+[whitepaper](./references/domas_breaking_the_x86_isa_wp.pdf).
+
+
+### Building
+
+Sandsifter requires first installing the Capstone disassembler:
+http://www.capstone-engine.org/
+
+Sandsifter can be built with:
+
+```
+make
+```
+
+and is then run with 
+
+```
+sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
+```
+
+### Flags
+
+Flags are passed to the sifter with --flag, and to the injector with -- -f.
+
+Example:
+
+```
+sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t
+```
+
+Sifter flags:
+
+--len - 
+	search for length differences in all instructions (instructions that
+	executed differently than the disassembler expected, or did not
+	exist when the disassembler expected them to
+
+--dis - 
+	search for length differences in valid instructions (instructions that
+	executed differently than the disassembler expected)
+
+--unk - 
+	search for unknown instructions (instructions that the disassembler doesn't
+	know about but successfully execute)
+
+--ill - 
+	the inverse of --unk, search for invalid disassemblies (instructions that do
+	not successfully execute but that the disassembler acknowledges)
+
+--tick - 
+	periodically write the current instruction to disk
+
+--save - 
+	save search progress on exit
+
+--resume - 
+	resume search from last saved state
+
+--sync - 
+	write search results to disk as they are found
+
+--low-mem - 
+	do not store results in memory
+
+Injector flags:
+
+-b -
+	mode: brute force
+
+-r -
+	mode: randomized fuzzing
+
+-t -
+	mode: tunneled fuzzing
+
+-d -
+	mode: externally directed fuzzing
+
+-R -
+	raw output mode
+
+-T -
+	text output mode
+
+-x -
+	write periodic progress to stderr
+
+-0 -
+	allow null dereference (requires sudo)
+
+-D -
+	allow duplicate prefixes
+
+-N -
+	no nx bit support
+
+-s seed -
+	in random search, seed value
+
+-B brute_depth -
+	in brute search, maximum search depth
+
+-P max_prefix -
+	maximum number of prefixes to search
+
+-i instruction -
+	instruction at which to start search (inclusive)
+
+-e instruction -
+	instruction at which to end search (exclusive)
+
+-c core -
+	core on which to perform search
+
+-X blacklist -
+	blacklist the specified instruction
+
+-j jobs -
+	number of simultaneous jobs to run
+
+-l range_bytes -
+	number of base instruction bytes in each sub range
+
+
+### Keys
+
+m: Mode - change the search mode (brute force, random, or tunnel) for the sifter
+
+q: Quit - exit the sifter
+
+p: Pause - pause or unpause the search
+
+
+### sudo
+
+For best results, the tool should be run as the root user.  This is necessary so
+that the process can map into memory a page at address 0, which requires root
+permissions.  This page prevents many instructions from segfaulting on memory
+accesses, which allows a more accurate fault analysis.
+
+
+### Legacy systems
+
+For scanning much older systems (i586 class processors, low memory systems),
+pass the --low-mem flag to the sifter and the -N flag to the injector:
+
+```
+sudo ./sifter.py --unk --dis --len --sync --tick --low-mem -- -P1 -t -N
+```
+
+If you observe your scans completing too quickly (for example, a scan completes
+in seconds), it is typically because these flags are required for the processor
+you are scanning.
+
+
+### README TODO
+
+* algorithms: random tunneling brute driven/mutator
+* detailed results enumeration
+* screenshots of bug types, final results
+* grep ./injector
+* 32 and 64 bit installs
+* prefixes and limitations
+* installing capstone help
+* terminal colors (export TERM='xterm-256color')
+* shrink screen to fit
+* example of targetted fuzzing
+
+
+### References
+
+* A whitepaper describing the approach is 
+  [here](./references/domas_breaking_the_x86_isa_wp.pdf).
+* Slides from the Black Hat 2017 presentation are
+  [here](./references/domas_breaking_the_x86_isa.pdf)

disas/capstone_32.py

@@ -0,0 +1,22 @@
+from capstone import *
+import sys
+from binascii import hexlify, unhexlify
+import os.path
+
+if os.path.exists(sys.argv[1]):
+    with open(sys.argv[1], 'r') as f:
+        byte_string = hexlify(f.read())
+else:
+    byte_string = sys.argv[1]
+
+md = Cs(CS_ARCH_X86, CS_MODE_32)
+try:
+    (address, size, mnemonic, op_str) = md.disasm_lite(unhexlify(byte_string), 0, 1).next()
+except StopIteration:
+    mnemonic="(unk)"
+    op_str=""
+    size = 0
+
+print "%s %s" % (mnemonic, op_str)
+print byte_string[:size*2]
+print "%d bytes" % size

disas/capstone_64.py

@@ -0,0 +1,22 @@
+from capstone import *
+import sys
+from binascii import hexlify, unhexlify
+import os.path
+
+if os.path.exists(sys.argv[1]):
+    with open(sys.argv[1], 'r') as f:
+        byte_string = hexlify(f.read())
+else:
+    byte_string = sys.argv[1]
+
+md = Cs(CS_ARCH_X86, CS_MODE_64)
+try:
+    (address, size, mnemonic, op_str) = md.disasm_lite(unhexlify(byte_string), 0, 1).next()
+except StopIteration:
+    mnemonic="(unk)"
+    op_str=""
+    size = 0
+
+print "%s %s" % (mnemonic, op_str)
+print byte_string[:size*2]
+print "%d bytes" % size

disas/disas_32.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+echo -ne `echo $1 | sed 's/\(..\)/\\\\x\1/g' ` > temp.bin
+
+echo
+echo "=== ndisasm ==="
+echo
+ndisasm -b32 temp.bin
+
+echo
+echo "=== objdump ==="
+echo
+objdump -D -b binary -mi386 temp.bin
+
+echo
+echo "=== capstone ==="
+echo
+python capstone_32.py $1
+
+rm temp.bin
+

disas/disas_64.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+echo -ne `echo $1 | sed 's/\(..\)/\\\\x\1/g' ` > temp.bin
+
+echo
+echo "=== ndisasm ==="
+echo
+ndisasm -b64 temp.bin
+
+echo
+echo "=== objdump ==="
+echo
+objdump -D -b binary -mi386 -Mx86-64 temp.bin
+
+echo
+echo "=== capstone ==="
+echo
+python capstone_64.py $1
+
+rm temp.bin
+