Self-Propelled Instrumentation

Note

1. Currently SPI depends on a customized dyninstAPI

• Build dyninst with arch-x86.h exposed to public
  • Move arch-x86.h from common/src to common/h
  • Replace #include "common/src/Types.h" with #include "dyntypes.h", and manually add the missing definitions to arch-x86.h
  • Add COMMON_EXPORT before class and function names inside arch-x86.h
  • Change #include "common/src/arch-x86.h" to #include "arch-x86.h" in the follwing files:
    • common/src/arch-x86.C
    • common/src/arch.h
    • instructionAPI/src/Instruction.C
    • instructionAPI/src/InstructionDecoder-x86.C
    • instructionAPI/src/Operation.C
  • Add #include "common/src/ia32_locations.h" to common/src/arch-x86.C
• Build dyninst with OpenMP disabled

2. Dynamically linked functions may appear twice during instrumentation, because these functions exist in .plt sections too.

TODOs

• Inter-host propel
• Solve the arch-x86.h dependency problem
  • A possible solution is to expose codegenAPI in dyninst
• OpenMP hangs during parsing
• Test suite
• Update MIST and SecStar

How to Install

1. Copy example-make.config to be config.mk.
2. Edit config.mk to define each macro variable.

• DYNINST_DIR: absolute path of Dyninst's lib directory
• SP_DIR: absolute path of this package (the output of pwd), because this file is also used by Makefile, I don't put pwd in it.
• DYNLINK: true for building shared library for agent, otherwise for building static library
• PLATFORM: set to x86_64-unknown-linux2.4 or i386-unknown-linux2.4

3. Run make spi in directory x86_64-unknown-linux2.4/ or i386-unknown-linux2.4/ to build injector and libagent.so.
4. Run make test_agent in one of the above directories to build example user agents
5. For more make options, see Make Arguments

How to Run

1. Export Runtime environment variables, see Environment Variable section for detail:
2. SP_DIR
3. PLATFORM
4. SP_AGENT_DIR
5. SP_DYNINST_DIR (DYNINST_ROOT/lib)
6. Make sure that your system does not block non-child ptrace

• To temporarily disable this measure (until a reboot), execute the following command:
  echo 0 > /proc/sys/kernel/yama/ptrace_scope
• To permanently disable this measure, edit the file /etc/sysctl.d/10-ptrace.conf and follow the directions in that file.
• For more information, see https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening

3. There are two ways to kick start the Self-Propel
   1. To use the environment variable LD_PRELOAD when starting the user process.
      Ex: LD_PRELOAD=$SP_DIR/PLATFORM/test_agent/print_test_agent.so [EXECUTABLE]
   2. To use the injector to force a running process to load agent library, note that injector has two modes, pid injection and port injection
      Ex: $SP_DIR/$PLATFORM/injector.exe pid [PID] or $SP_DIR/$PLATFORM/injector.exe port [PORT NUMBER]

Interprocess Propel

• Local Machine
  • Interprocess propelling relies on the following environment variables: SP_DIR, PLATFORM, SP_AGENT_DIR
• Inter-host
  • Export desired environment variables in your .bashrc file. Following environment variables are necessary:
    • SP_DIR
    • PLATFORM
    • SP_AGENT_DIR
    • SP_DYNINST_DIR (DYNINST_ROOT/lib)
    • Add SP_DYNINST_DIR, SP_DIR/PLATFORM to your LD_LIBRARY_PATH

Environment Variables

1. FOR DEBUGGING
2. SP_COREDUMP: enables core dump when segfault happens
3. SP_DEBUG: enables printing out debugging messages
4. SP_FDEBUG: enables printing debugging messages to an output file
5. SP_TEST_RELOCINSN: only uses instruction relocation instrumentation worker
6. SP_TEST_RELOCBLK: only uses call block relocation instrumentation worker
7. SP_TEST_SPRING: only uses sprint block instrumentation worker
8. SP_TEST_TRAP: only uses trap instrumentation worker
9. SP_NO_TAILCALL: don't instrument tail calls
10. SP_LIBC_MALLOC: will always use libc malloc
11. SP_NO_LIBC_MALLOC: will never use libc malloc
12. SP_DEBUG_$TYPE: Enables debugging for different components of SPI. Options are SP_DEBUG_INJECTOR, SP_DEBUG_COMMON, SP_DEBUG_PATCHAPI, SP_DEBUG_IPC, SP_DEBUG_WORKER, SP_DEBUG_SIGTRAP, SP_DEBUG_AGENT. Each option corresponds roughly to directories under /src.
13. SP_TIMING: Enables timestamps on debug output messages, in seconds.
14. FOR RUNTIME
15. PLATFORM: 'i386-unknown-linux2.4' for x86 or 'x86_64-unknown-linux2.4' for x86-64
16. SP_DIR: the root directory of self-propelled instrumentation.
17. SP_AGENT_DIR: the directory path of agent shared library that will be injected.
18. SP_DYNINST_DIR: the directory path of the modified Dyninst build's /lib dir

Refer to template.sh in this repository for an example script to configure/run with SPI.

Shared memory id used

1. 1986+[user_process_pid]: for communication between injector process and user process

Make Arguments

For testing

• make unittest: build unittests
• make mutatee: build simple mutatees
• make external_mutatee: build real world mutatees
• make test: unittest + mutatee + external_mutatee

For main self propelled

• make injector_exe
• make agent_lib
• make spi: agent_lib + injector_exe

For everything

• make / make all: spi + test

For cleaning

• make clean_test: clean test stuffs
• make clean: only clean core self-propelled stuffs, excluding dependency
• make clean_all: clean everything, including dependency
• make clean_objs: clean core self-propelled objs