[Spike] Scale OSSF Scorecards prescriptions out of GitHub for aggregation by revision #31968
Labels
kind/feature
Categorizes issue or PR as related to a new feature.
priority/important-soon
Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
sig/stack-guidance
Categorizes an issue or PR as relevant to SIG Stack Guidance.
Comments
mayaCostantini
added
the
kind/feature
|
/sig stack-guidance |
sesheta
added
sig/stack-guidance
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is your feature request related to a problem? Please describe.
As we will start aggregating Scorecards prescriptions by project repository revision as present in the new
scorecards-v2BigQuery dataset and possibly create those prescriptions for packages from other ecosystems, we should think about a more scalable solution to have this data available.The current size of the
prescriptionsdataset is currently of ~500M, which will largely exceed the recommended GitHub limit of 5GiB for a repository and cause storage and performance issues.Describe the solution you'd like
Set up a new database (possibly non-relational) or make new Scorecards prescriptions available in a S3 bucket accessed through a webservice.
Additional context
Related to thoth-station/core#440
The text was updated successfully, but these errors were encountered: