A runtime system to deterministically record lock acquires and replay them on an arbitrary number of cores. If you know your program has an atomicity bug, this system allows you to record execution repeatedly until you see the failure. Then you can replay the system (possibly in a debugger) on as many cores as you like, achieving good speedup if your program has ample parallelism. Our system ensures weak determinism during replay. Specifically, lock acquisitions occur in the same order as recording. You can find more details in the PPoPP 2017 paper Processor-Oblivious Record and Replay.
Currently, you need to build the library and link it statically, although there is no technical reason why this couldn't be a dynamically-loaded library. This library relies on modifications to the Cilk runtime library (provided in the cilkrtssuspend directory), as well as a special compiler that won't inline some Cilk helper functions. You can find such a compiler here.
The code in this repository is licensed under The MIT License (see LICENSE) except for the runtime modifications in cilkrtssupend, which is separately licensed. Note that some scripts in this repository access other open source projects, which are licensed separately.
A relatively recent version of Linux is recommended, with a C++11 version of the C++ standard library. PORRidge was tested on Ubuntu 16.04, but is expected to run correctly in other Linux distributions.
To reproduce performance results, Google's tcmalloc should be
installed. Also, the GNU gold linker should be installed as
ld. This requirement may be eliminated as the project is
refactored. The benchmark script requires GNU datamash. Compiling
the dedup and ferret benchmarks requires additional
dependencies. The ones I can remember are libssl-dev libgsl-dev ligjpeg62-dev libbz2-dev
Depending on the version of g++ you have installed you may need to
edit the CPLUS_INCLUDE_PATH variable at the top of setup.sh. Our
(very old) version of llvm/clang does not like the newer stdlib
headers that come with g++-7 (and possibly g++-6, I haven't
tested). So I have installed g++-5 and forced clang to use that
include path.
The script setup.sh should build the modified compiler, the modified Cilk Plus runtime, and all benchmarks.
Before running, make sure to install GNU gold as your linker -- the system ld should point to gold. Installing gold also installs a header called plugin-api.h, usually in either /usr/include or /usr/local/include. Find this file and replace the BINUTILS_PLUGIN_DIR variable in setup.sh with this path.
Some of the benchmarks require many mmap calls when replaying on large data sets. To account for this, you may need to change the number of mmap calls allowed, e.g.:
sudo sysctl -w vm.max_map_count=1000000
The bench/bench.sh script will run all benchmarks using PORRidge. You can change the variable CORES to a list of cores to run the benchmarks on. By default it runs on 1,2,4,8,12, and 16 cores. The results are printed directly to the screen by default.
Currently, the provided interface is basically that of pthread_spinlock, so just replace calls to pthread_spinlock_ with porr::spinlock::. That namespace will probably change.
By default, once you've linked, nothing will actually happen. You'll need to set the environment variable PORR_MODE to tell the library to either record or replay. You can record with
PORR_MODE=record ./prog
which will write out the results to ".cilkrecord". To replay, use
PORR_MODE=replay ./prog
You can use the PORR_FILE environment variable to change this filename used by both recording and replaying.
If you dynamically allocate memory for locks, you should call the
porr::reserve_locks(<num locks>)
function beforehand. For any cilk_for loops you should also manually specify the cilk grainsize, e.g.:
#pragma cilk grainsize = 1024
These restrictions allowed for an easier implementation; they are not inherent to the design of PORRidge.
Please use the following citiation when using this runtime in your work:
Robert Utterback, Kunal Agrawal, I-Ting Angelina Lee, and Milind Kulkarni, "Processor-Oblivious Record and Replay", Proceedings of the 22Nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP), 2017. Available: https://dl.acm.org/citation.cfm?id=3018764.
This research was supported in part by National Science Foundation grants CCF-1150036, CCF-1218017, XPS-1439062, CCF-1150013, CCF-1439126, CCF-1527692, and Department of Energy grant DE-SC0010295.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation and Department of Energy.