Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initializer job name in Helm chart uses date which prevent application in ArgoCD from being in Sync #11039

Open
Brawdunoir opened this issue Oct 10, 2024 · 2 comments

Comments

@Brawdunoir
Copy link

Hi,

I’m using the Helm chart version 1.6.153

Is your feature request related to a problem? Please describe
Because of the helper initializer.jobname, a new job could be created each minute.
This results in a permanent OutOfSync status of the defectdojo application within Argo CD, with a missing job ressource.

Describe the solution you'd like
One solution could be to use Helm hooks like pre-install and/or pre-upgrade and always use the same job name (thanks to hook deletion policies).

Additional context
Helper causing the problem:

{{- define "initializer.jobname" -}}
{{ .Release.Name }}-initializer-{{- printf "%s" now | date "2006-01-02-15-04" -}}
{{- end -}}

Argo CD dashboard:
{7743EE4A-399F-49EA-B326-B34A18A33623}

@kiblik
Copy link
Contributor

kiblik commented Oct 11, 2024

Thank you, @Brawdunoir, for pointing this out. I had planned to look closer at my similar/related issues, and you probably found the reason.

Unfortunately, pre- hooks would not work well for new installations. The job would be triggered before the database would be started. The database would not be started until the job was finished (as far as I know - we have been solving a similar issue with post- hooks).

Still not sure what is the best solution but "same name" sounds like a suitable solution.

I would like to check it deeper.

Sidenote: this is a side-effect of how Argo handles charts. As far as I know, there is no issue for "regular" helm deployments right now.

@Brawdunoir
Copy link
Author

Thanks for the quick response! I have quite the experience of building charts and got into this job issue lately but without your chart dependencies constraint (we use Helmfile to deploy/order multiple charts).

Reusing the same job name without hooks will not work because a Job has a lot of immutable fields (including pod labels and container images). That's the reason behind the hook deletion policies.

It would be interesting to understand how other charts like Bitnami's are copping with this issue.

I suspect that because (in my knowledge) a proper mechanism from Helm itself does not exist to handle such situations, you could only rely on an init-container on your job.
Your application will still be launched, without prior initialization of the database, but hey, that's life with Kubernetes, and your application should handle such cases programmatically (with checks and retries). And after maybe a few restarts because of readiness, at the end, everything should stabilize.

Using regular Helm deployment, are the previous jobs cleaned on each upgrade?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants