Add/cle facts (#13)

* adding smeagle. Would be good to have others look at what we should do for callsites, and if it is okay to do the same thing for variables as functions
* just run libabigail and spack tests
* only include linked deps in experiment
* add traceback print exceptions in spack experiment
* ensuring we install deprecated and skip libabigail / predictions
* dont disable libabigail, etc
* adding example smeagle facts generator for Matt;
* config command borked the config, tell user to do it manually
* do not alter spack config it gets borked
* start of work to update facts
* pair programming with tim to fix locations, etc. Will add automated tests next

* Add/asp tests (#14)
* adding atoms and tests
* add workflow dispatch
* missing Makefile
* ensure we install spliced
* ensure we test truth facts if they exist
* fixing bug with fact parsing structs with pointer
* fixing recursive types fact generation
* fixing double indirections (pointers and references w/ typedefs
* fixing struc w/ struct
* adding more support for arrays
* adding offsets to var
* add spliced using cle
* prepare to run cle in spliced
* refactor of smeagle to use elfcall and new algorithm! I still need to refactor the other predictors, likely tomorrow.
* preparing to test libabigail since my local spack is entirely borked. Because.
* re-enable smeagle
* updating spliced to have compatible model for elfcall idea
* account for spack ld library paths
* set different_libs given mock splice
* change cle location

Signed-off-by: vsoch <[email protected]>

.github/workflows/test-atoms.yaml

@@ -0,0 +1,26 @@
+name: Atoms (ASP) Generation Tests
+on: 
+  workflow_dispatch:
+  pull_request: []
+
+jobs:
+  test-basic:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3        
+      with:
+        submodules: recursive
+    - name: Install Library
+      run: |
+        # archinfo, pyvex, pyelftools, then cle
+        pip install git+https://github.com/angr/archinfo
+        pip install git+https://github.com/angr/pyvex
+        pip install git+https://github.com/eliben/pyelftools
+        pip install git+https://github.com/vsoch/cle
+        pip install pytest deepdiff
+        pip install -e .
+
+    - name: Run Examples
+      run: |
+        docker run -t -v $PWD:/code gcc:12.1 bash -c "cd /code && make"
+        pytest -xs tests.py

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "examples/smeagle"]
+	path = examples/smeagle
+	url = https://github.com/buildsi/smeagle-examples

CHANGELOG.md

@@ -14,6 +14,7 @@ and **Merged pull requests**. Critical items to know are:
 The versions coincide with releases on pip. Only major versions will be released as tags on Github.
 
 ## [0.0.x](https://github.com/buildsi/spliced/tree/main) (0.0.x)
+ - add smeagle predict (via singularity container) (0.0.16)
  - libabigail should also run abidiff (0.0.15)
  - add spack test as a predictor for spack installs (0.0.14)
  - abicompat is missing the binary as the first argument! (0.0.13)

Makefile

@@ -0,0 +1,10 @@
+LIBS = ./examples/smeagle
+
+all: compile
+
+compile: $(LIBS)/*
+	for dir in $^ ; do \
+		echo "Building" $${dir} ; \
+		$(MAKE) -C $${dir}; \
+	done
+

README.md

@@ -10,6 +10,10 @@
 
 This library is written and maintained by [@vsoch](https://github.com/vsoch)
 
+## TODO
+
+ - Can we (and do we want to) run in parallel?
+
 ## License
 
 Spack is distributed under the terms of both the MIT license and the

action/install/scripts/install.sh

@@ -15,6 +15,18 @@ fi
 
 python -m pip install --upgrade pip setuptools wheel
 
+# These are required deps for cle
+git clone https://github.com/vsoch/cle /tmp/cle
+cd /tmp/cle
+
+# archinfo, pyvex, pyelftools, then cle
+python -m pip install git+https://github.com/angr/archinfo
+python -m pip install git+https://github.com/angr/pyvex
+python -m pip install git+https://github.com/eliben/pyelftools
+python -m pip install .
+cd ../
+rm -rf /tmp/cle
+
 # Case 1: no branch or release, install from pip
 if [ -z "${INPUT_BRANCH}" ] && [ -z "${INPUT_RELEASE}" ]; then
     printf "Installing latest from pypi\n"

docs/getting_started/developer-guide.rst

@@ -8,8 +8,32 @@ This developer guide includes more complex interactions like adding experiments
 or using containers. If you haven't read :ref:`getting_started-installation`
 you should do that first.
 
+Design of Spliced
+=================
+
+As a reminder, spiced works by having **experiment runners** (e.g., spack) that present a splice object with metadata about libraries to a **predictor** to parse (and make predictions for). This design section will include details for how the experiment runner and subsequent predictors work.
+
+Spack Experiment Runner
+-----------------------
+
+The spack experiment runner works by starting with a main library (e.g., caliper) and a dependency of interest (e.g., papi). We then, for each version of the depedency:
+
+1. Perform a splice of the new version into the main package to derive a new spliced directory.
+2. Find libs and binaries within the new (spliced) package install (and the original)
+3. Use `Elfcall <https://vsoch.github.io/elfcall>`_ to emulate the linker and find paths of libraries with needed (undefined) symbols.
+4. Present the elfcall output to each predictor to use appropriately.
+
+In the case of a "diff" comparator like libabigail or symbolator, we match names of libraries that are used for each of the original dependency (A) and spliced dependency (B) case.
+
+In the case of smeagle, we first check if elfcall didn't find any symbols, and if so we stop and report failure. The smeagle model depends on function symbol names and will always fail if a symbol is entirely missing. If we found all our symbols, we then
+derive first smeagle json (saved to a cache based on the library name) and 
+then a scoped set of asp facts the main library (A) and the entire set of dependencies (under the namespace B) to compare to. Since elfcall only matches one library to one symbol, there won't be conflicts within the B space.
+
+**Note** that elfcall can only parse ELF. A library or binary that is not ELF cannot be included, and this is a small set but still represents a limitation of the analysis.
+
+
 Add An Experiment
-==================
+=================
 
 The core of an experiment is to be able to run the initial steps for a splice,
 and return the splice object, which should have binaries and libraries for a spec pre and post splice,
@@ -20,12 +44,11 @@ set them arbitrarily to our liking).
 Add A Predictor
 ===============
 
-A predictor should be added as a module to [spliced/predict](spliced/predict) so it is retrieved
+A predictor should be added as a module to ``spliced/predict`` so it is retrieved
 on init. It should have a main function, predict, which takes a splice object and optional kwargs.
-At this point you can iterate through the splice structure to use whatever metadata you need. E.g.,:
+At this point you can iterate through the splice structure to use whatever metadata you need. The splice metadata is derived from `Elfcall _<https://vsoch.github.io/elfcall>`_ which should have all the sets of libraries found and symbols for each (and we stop when all needed and undefined symbols are found). E.g.,:
 
- - splice.libs: is a dictionary with "original" and "spliced" for original and spliced libs, respectively
- - splice.binaries: is the same structure, but with binaries for the original and spliced package
+## TODO need to update here
 
 Importantly, your predictor should set `spliced.predictions[<name_of_predictor>]` to be a list of dictionaries,
 where you can put any needed metadata. The binary/lib is suggested, along with a return code or message from the console,

docs/getting_started/installation.rst

@@ -16,6 +16,23 @@ under development) you need to clone a branch from vsoch.
     $ source spack/share/spack/setup-env.sh
 
 
+Development Dependencies
+========================
+
+The smeagle predictor currently requires cle:
+
+.. code:: console
+
+    git clone https://github.com/vsoch/cle /tmp/cle
+    cd /tmp/cle
+    
+    # archinfo, pyvex, pyelftools, then cle
+    pip install wheel
+    pip install git+https://github.com/angr/archinfo
+    pip install git+https://github.com/angr/pyvex
+    pip install git+https://github.com/eliben/pyelftools
+    pip install .
+
 Pypi
 ====
 

examples/asp/A.asp

@@ -0,0 +1,20 @@
+
+%============================================================================
+% Library Facts
+%============================================================================
+
+%----------------------------------------------------------------------------
+% Library: /home/vanessa/Desktop/Code/compspec/compspec-python/examples/dwarf/lib/libmath/v2/lib.v2.so
+%----------------------------------------------------------------------------
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic3AddEii","Import","Integer32","%rdi").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic3AddEii","Import","Integer32","%rsi").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic3AddEii","Export","Integer32","%rax").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8SubtractEdd","Import","Float64","%xmm0").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8SubtractEdd","Import","Float64","%xmm1").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8SubtractEdd","Export","Float64","%xmm0").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Import","Float64","%xmm0").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Import","Float64","%xmm1").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Export","Float64","%xmm0").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic6DivideEdd","Import","Float64","%xmm0").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic6DivideEdd","Import","Float64","%xmm1").
+abi_typelocation("a","_ZN11MathLibrary10Arithmetic6DivideEdd","Export","Float64","%xmm0").

examples/asp/B.asp

@@ -0,0 +1,20 @@
+
+%============================================================================
+% Library Facts
+%============================================================================
+
+%----------------------------------------------------------------------------
+% Library: /home/vanessa/Desktop/Code/compspec/compspec-python/examples/dwarf/lib/libmath/v1/lib.v1.so
+%----------------------------------------------------------------------------
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic3AddEdd","Import","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic3AddEdd","Import","Float64","%xmm1").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic3AddEdd","Export","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8SubtractEdd","Import","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8SubtractEdd","Import","Float64","%xmm1").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8SubtractEdd","Export","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Import","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Import","Float64","%xmm1").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic8MultiplyEdd","Export","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic6DivideEdd","Import","Float64","%xmm0").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic6DivideEdd","Import","Float64","%xmm1").
+abi_typelocation("b","_ZN11MathLibrary10Arithmetic6DivideEdd","Export","Float64","%xmm0").

examples/asp/README.md

@@ -0,0 +1,131 @@
+# Spliced to print Asp
+
+## Single Facts
+
+This is an example of using spliced just to print asp facts to the terminal,
+which is an easy thing it can do! We start with [smeagle-output.json](smeagle-output.json),
+load it, and then use the same SmeagleRunner to print facts to the terminal.
+
+```python
+import spliced.utils as utils
+import spliced.predict.smeagle as smeagle
+
+cli = smeagle.SmeagleRunner()
+data = utils.read_json("smeagle-output.json")
+
+# We can accept a path (will run smeagle) or the raw data, so
+# it is important to provide a kwarg here!
+cli.generate_facts(data=data)
+```
+```bash
+%============================================================================
+% Library Facts
+%============================================================================
+
+%----------------------------------------------------------------------------
+% Library: libtest.so
+%----------------------------------------------------------------------------
+abi_typelocation("libtest.so","_Z7bigcallllllln","a","Integer","%rdi","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","b","Integer","%rsi","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","c","Integer","%rdx","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","d","Integer","%rcx","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","e","Integer","%r8","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","f","Integer","framebase+8","import","0").
+```
+
+This is provided in a little [generate-facts.py](generate-facts.py) courtesy script.
+
+```bash
+$ python generate-facts.py smeagle-output.json
+```
+```bash
+%============================================================================
+% Library Facts
+%============================================================================
+
+%----------------------------------------------------------------------------
+% Library: libtest.so
+%----------------------------------------------------------------------------
+abi_typelocation("libtest.so","_Z7bigcallllllln","a","Integer","%rdi","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","b","Integer","%rsi","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","c","Integer","%rdx","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","d","Integer","%rcx","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","e","Integer","%r8","import","0").
+abi_typelocation("libtest.so","_Z7bigcallllllln","f","Integer","framebase+8","import","0").
+```
+
+
+## Simulating a Solve
+
+### Stability Test
+If you want to simulate a stability test, first run the script, providing two libs as "A" (original)
+and "B" (comparison). Here are examples (both ways) with our math library:
+
+```bash
+$ python stability-test.py ../../../compspec/compspec-python/examples/dwarf/lib/libmath/v1/lib.v1.so  ../../../compspec/compspec-python/examples/dwarf/lib/libmath/v2/lib.v2.so
+
+Missing Imports
+---------------
+ _ZN11MathLibrary10Arithmetic3AddEdd Float64 %xmm0
+ _ZN11MathLibrary10Arithmetic3AddEdd Float64 %xmm1
+
+Missing Exports
+---------------
+ _ZN11MathLibrary10Arithmetic3AddEdd Float64 %xmm0
+```
+```bash
+$ python stability-test.py ../../../compspec/compspec-python/examples/dwarf/lib/libmath/v2/lib.v2.so  ../../../compspec/compspec-python/examples/dwarf/lib/libmath/v1/lib.v1.so
+
+Missing Imports
+---------------
+ _ZN11MathLibrary10Arithmetic3AddEii Integer32 %rdi
+ _ZN11MathLibrary10Arithmetic3AddEii Integer32 %rsi
+
+Missing Exports
+---------------
+ _ZN11MathLibrary10Arithmetic3AddEii Integer32 %rax
+```
+
+This will generate A.asp and B.asp. To run this more manually with clingo, you can copy your stability model here:
+
+```bash
+$ cp ../../spliced/predict/smeagle/lp/stability.lp .
+```
+and run these in a clingo container (or via a local clingo install) like:
+
+```bash
+$ docker run -v $PWD:/data --rm -it ghcr.io/autamus/clingo bash
+$ cd /data
+$ clingo --out-ifs=\\n A.asp B.asp stability.lp
+```
+And you'll get the same answer:
+
+```bash
+clingo version 5.5.1
+Reading from A.asp ...
+Solving...
+Answer: 1
+missing_exports("_ZN11MathLibrary10Arithmetic3AddEii","Integer32","%rax")
+missing_imports("_ZN11MathLibrary10Arithmetic3AddEii","Integer32","%rdi")
+missing_imports("_ZN11MathLibrary10Arithmetic3AddEii","Integer32","%rsi")
+SATISFIABLE
+
+Models       : 1
+Calls        : 1
+Time         : 0.003s (Solving: 0.00s 1st Model: 0.00s Unsat: 0.00s)
+CPU Time     : 0.003s
+```
+
+### Compatible Test
+
+This is the test that smeagle is doing for our experiment, since we are loading
+all symbols within a space for a library A and dependencies (B) (to the scoped set we
+can find).
+
+
+```bash
+$ docker run -v $PWD:/data --rm -it ghcr.io/autamus/clingo bash
+$ cd /data
+$ clingo --out-ifs=\\n ccache-swig.asp compatible.lp 
+```
+