From fa8cb4389256aa16d09562c43b1a181b9b7fa3e2 Mon Sep 17 00:00:00 2001 From: Louis Cabrol Date: Mon, 6 Feb 2023 15:34:05 +0100 Subject: [PATCH 1/2] fix: add link to HIP-0018 Signed-off-by: Louis Cabrol --- hips/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/hips/README.md b/hips/README.md index 796ad06d..9b1124fe 100644 --- a/hips/README.md +++ b/hips/README.md @@ -26,3 +26,4 @@ restricted markdown format and can be found in the - [hip-0014: Helm Triage Maintainers](hip-0014.md) - [hip-0015: Annotation for Images used by Helm Charts](hip-0015.md) - [hip-0017: Helm OCI MediaType Registration](hip-0017.md) +- [hip-0018: Add Repository URL and tarball digest to a chart's release metadata](hip-0018.md) From 40c8a7ed09a9126ee29e5a2a98db9d6543742d86 Mon Sep 17 00:00:00 2001 From: Louis Cabrol Date: Mon, 6 Feb 2023 18:30:01 +0100 Subject: [PATCH 2/2] feat: hip-0019 split manifest proposal Signed-off-by: Louis Cabrol --- hips/hip-0019.md | 65 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 hips/hip-0019.md diff --git a/hips/hip-0019.md b/hips/hip-0019.md new file mode 100644 index 00000000..3044f5c0 --- /dev/null +++ b/hips/hip-0019.md @@ -0,0 +1,65 @@ +--- +hip: "0019" +title: "Secrets storage driver support for split manifests" +authors: [ "Louis Cabrol" ] +created: "2023-02-06" +type: "feature" +status: "draft" +--- + +## Abstract + +Helm, by default in version 3, stores its Release manifest in a Secret object in the target Kubernetes namespace, after a double base64 encoding + GZIP compression. + +This object type, [by Kubernetes design](https://kubernetes.io/docs/concepts/configuration/secret/#restriction-data-size) has a maximum size allowed for data restricted to 1Mib. + +This imposes a limit on the maximum number of templates a Helm Release can manage, as well as limits the overall complexity a Helm Chart can reach before exceeding this limit. + +## Motivation + +This has been reported a number of times ([here](https://github.com/helm/helm/issues/10986), [here](https://github.com/helm/helm/issues/8281), and many others within those two links). As Charts keep growing in complexity through modular, optional components, and Releases keep growing in the number of Kubernetes objects they manage, more and more people will keep hitting this limit. + +Deploying a Chart that exceeds this preset size currently yields a Kubernetes error, without first warning the user that the Release Manifest size exceeds the maximum, which isn't a good user experience. + +My specific motivation for this fix is a complex, modular Helm Chart that deploys Grafana dashboards. As such, it contains (many) dashboard templates as JSON that are templated through Helm into ConfigMap objects. Those template files cannot be `helmignored`, and their cumulated size alone is close to the Secret size limit, before encoding and compression. As such, I'm starting to run into issues when deploying to complex environments that require a lot of those dashboard definitions, which could be solved by splitting the chart among different sub-charts and/or sub-releases, but this would be working around the problem, and not adressing it. + +## Specification + +The proposal is to create a new "type" of Secret, `helm.sh/partial.v1`, that would enable the Release Manifest to be split across multiple secrets as needed, named after the following format `sh.helm.partial.v1..v-`. The presence of further "partial" secrets would then be indicated through an optional `continuedIn` label. Changes should only be made within the Secrets storage driver, so as to retain the current driver interface. + +## Backward compatibility + +Containing changes to the Secrets storage driver means there's no compatibility issue with other storage types. + +For Secrets, marking the Secrets with `helm.sh/partial.v1` for partial elements means identifying releases still only relies on the presence of secrets that are named after this format `sh.helm.release.v1..v` with the `helm.sh/release.v1` type. + +Older versions of Helm would not be able to interact with the new split manifests in most cases. + +## Rationale + +The only storage type that contains this hardcoded limit is Secrets. ConfigMap size can be increased at the `etcd` level, and the SQL driver doesn't have such limitations by design. + +This expansion to the Secrets spec allows users not to maintain a separate PSQL database to store Helm release manifests, instead allowing them to make use of the internal components Kubernetes provides, while maintaining backwards compatibility with current releases. + +## How to teach this + +This size limit currently isn't well documented within Helm, and only found through the aforementioned issues I linked to, so there isn't much to teach. There would not be any user-facing changes, barring the ability to deploy larger charts. + +## Reference implementation + +[Helm PR](https://github.com/helm/helm/pull/11791) + +## Rejected ideas + +None as far as I know + +## Open issues + +None as far as I know + +## References + +- [Kubernetes Secrets data size restrition](https://kubernetes.io/docs/concepts/configuration/secret/#restriction-data-size) +- Issues: + - [An issue discussing this problem](https://github.com/helm/helm/issues/10986) + - [Another issue discussing this problem](https://github.com/helm/helm/issues/8281)