Skip to content

Project Repository Structure

Jump to bottom
Peter Matula edited this page Jan 15, 2018 · 7 revisions

Present State

After initial struggles and mistakes (see Past States), we settled on the following structure of the RetDec project repositories:

  • Sources of all the libraries and tools that make up RetDec are in a single repository called retdec.
  • Sources of our tools are in the directory src. Headers in include/retdec.
  • All the third-party libraries are in the directory deps.
  • Sources of the third-party libraries are not in the retdec repository. They are downloaded via CMake's external project mechanism.
  • Preferably, we get third-party libraries directly from their official repositories.
  • However, if we use modified third-party libraries, we get them from our modified forks in avast-tl.
  • There are no git submodules, C++ dependency managers like Conan, etc. Only CMake external projects.
  • (TODO: not yet implemented) It is possible to build, install, and use only the selected subset of RetDec components. E.g. only fileformat library and fileinfo tool.
  • (TODO: not yet implemented) All the components support a proper system-wide installation (includes, dependencies, documentation, etc.). After the installation, libraries can be used in a standard way.
  • (TODO: not yet implemented) If RetDec uses unmodified third-party library, and this library (in a required version) is installed in the system, it is used instead of downloading and building it from sources.

Advantages

This solution has the following advantages:

  • Simple. No git submodules, C++ dependency managers, etc. Only CMake external projects.
  • Development-friendly. A single repository solution is the fastest and most comfortable option for development.
  • Clean. CMake external projects keep the third-party sources out of the retdec repository and provide an elegant way to reference particular versions used by RetDec.
  • (TODO: not yet implemented) Build system preserves the advantages of independent repositories - selective build and installation.

Disadvantages

This solution has the following disadvantages:

  • If someone is interested only in some component of the RetDec project (e.g. fileformat), he/she needs to clone an entire repository containing all the RetDec libraries and tools, and configure CMake to build and install only the desired component. This is less elegant than cloning only the desired independent repository (e.g. fileformat).

Past States

How did we come to this:

  • Originally (before open-sourcing), everything was in one giant internal repository (including sources of third-party projects, regression tests, etc.). It was messy but easy to develop.
  • Before open-sourcing, we wanted to split the repository into its logical parts, so that they can be built, used, and developed independently.
  • We decided to create a separete git repository for every logical component and link them together via git submodule mechanism. The dependency tree looked like this: Graph of git submodules dependencies
  • This proved confusing (#72), hard to maintain, slow to develop, and exhibited several other issues like #14, #48.
  • Especially, the git submodule mechanism proved to be extremely irritating.
  • Instead of reducing the complexity of the dependency tree, or complicating the matter by using C++ dependency managers, we decided to go the LLVM way and just put all the related libraries and tools together in a single repository.
  • (TODO: not yet implemented) We keep the advantage of independent repositories by implementing the build system in a way, which allows build and installation of only the user-selected components.
Clone this wiki locally