This pipeline assesses the ability of SBOMqs and SBOM-scorecard to detect changes made to SBOMs.
The pipeline consists of major stages:
- SBOM generation
- compliance result generation
- Data aggregation.
- SBOMqs SBOMqs installation instruction
- Trivy Trivy installation instructions
- Syft Syft installation instructions
When the generate_sbom.sh is run the 02_generate/04_product directory is populated with a repository of SBOMs created with the generation tools Syft and Trivy. The SBOMs are generated from the top 100 docker images for each of these major images an SBOM is generated in CDX and SPDX by both Syft and Trivy for up to 100 previous versions. The number of versions can be changed in the generate_sbom.sh by changing the -v argument to the new number of versions.
By running the create_tool_outputs.sh the 03_evaluate/04_product directory will be populated with 16 subdirectories that contain the outputs of SBOM-scorecard and SBOMqs. Each SBOM from the repository has each of the 7 NTIA minimum elements removed according to the -n arg in create_tool_outputs.sh with replacement (see diagram) and is run through SBOMqs and SBOM-scorecard. These outputs are from whole SBOMs and Hollowed SBOMs the hollowing can be controlled in the create_tool_outputs.sh by changing the num variable. The default is for up to 5 instances of each minimum element to be removed.
By running the data_frame.sh the JSON produced in the previous step is parsed into CSV files, one for each of the directories in 03_evaluate/04_product.