Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gateway2: Add readiness and liveness probes to the kube gloo gateway pod #10332

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

davidjumani
Copy link
Contributor

@davidjumani davidjumani commented Nov 13, 2024

Description

This update allows users to enable and add custom readiness and liveness probes to the Kubernetes Gloo Gateway.
This allows zero downtime when upgrading gloo or when rolling out new pods

API changes

The following helm changes have been made :

  • kubeGateway.gatewayParameters.glooGateway.podTemplate.terminationGracePeriodSeconds to specify the terminationGracePeriodSeconds.
  • kubeGateway.gatewayParameters.glooGateway.podTemplate.gracefulShutdown to configure the graceful shutdown config for the envoy container.
  • kubeGateway.gatewayParameters.glooGateway.podTemplate.customLivenessProbe to specify a custom liveness probe for the envoy container.
  • kubeGateway.gatewayParameters.glooGateway.podTemplate.livenessProbeEnabled to enable the liveness probe. If the customLivenessProbe is not specified, a default liveness probe is set.
  • kubeGateway.gatewayParameters.glooGateway.podTemplate.customReadinessProbe to specify a custom readiness probe for the envoy container.
  • kubeGateway.gatewayParameters.glooGateway.podTemplate.probes to enable the readiness probe. If the customReadinessProbe is not specified, a default readiness probe is set.

The following fields have been added to the GatewayParameters.Kube.PodTemplate field :

  • GracefulShutdown
  • TerminationGracePeriodSeconds
  • ReadinessProbe
  • LivenessProbe

Code changes

Updated the deployer to read the new values from the gateway params and generate the appropriate helm values for the gloo gateway pod deployment

CI changes

Added the zero downtime config to all Kubernetes tests to ensure this does not break an existing feature

Docs changes

TODO in a followup with @solo-io/solo-docs

Context

https://github.com/solo-io/solo-projects/issues/7084

Interesting decisions

The helm values were created to mimic the existing values to setup the probes and zero downtime config for edge classic. Ref: https://github.com/solo-io/solo-projects/issues/7084

Testing steps

Added unit tests

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works

@davidjumani davidjumani requested a review from a team as a code owner November 13, 2024 16:53
@github-actions github-actions bot added keep pr updated signals bulldozer to keep pr up to date with base branch work in progress signals bulldozer to keep pr open (don't auto-merge) labels Nov 13, 2024
@davidjumani davidjumani removed the keep pr updated signals bulldozer to keep pr up to date with base branch label Nov 13, 2024
@davidjumani davidjumani force-pushed the add-readiness-liveness-probe branch from 74b3fa3 to debf04a Compare November 13, 2024 16:53
@davidjumani davidjumani changed the title Add readiness liveness probe gateway2: Add readiness liveness probes to the kube gloo gateway pod Nov 13, 2024
Copy link

github-actions bot commented Nov 13, 2024

Visit the preview URL for this PR (updated for commit 31a2e5d):

https://gloo-edge--pr10332-add-readiness-livene-dgc4zcrn.web.app

(expires Wed, 20 Nov 2024 20:39:56 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: 77c2b86e287749579b7ff9cadb81e099042ef677

@solo-changelog-bot
Copy link

Issues linked to changelog:
https://github.com/solo-io/solo-projects/issues/7084

@davidjumani davidjumani added keep pr updated signals bulldozer to keep pr up to date with base branch and removed work in progress signals bulldozer to keep pr open (don't auto-merge) labels Nov 13, 2024
@davidjumani davidjumani changed the title gateway2: Add readiness liveness probes to the kube gloo gateway pod gateway2: Add readiness and liveness probes to the kube gloo gateway pod Nov 13, 2024
Comment on lines +101 to +111
{{- else }}
exec:
command:
- wget
- -O
- /dev/null
- 127.0.0.1:19000/server_info
initialDelaySeconds: 3
periodSeconds: 10
failureThreshold: 3
{{- end }}{{/*if $gg.podTemplate.customLivenessProbe*/}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{{- else }}
exec:
command:
- wget
- -O
- /dev/null
- 127.0.0.1:19000/server_info
initialDelaySeconds: 3
periodSeconds: 10
failureThreshold: 3
{{- end }}{{/*if $gg.podTemplate.customLivenessProbe*/}}
{{- end }}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This provides a default probe if a custom one is not specified

// If specified, the pod's graceful shutdown spec.
//
// +kubebuilder:validation:Optional
GracefulShutdown *GracefulShutdownSpec `json:"gracefulShutdown,omitempty"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we're adding new fields to PodTemplate we also need to add them to the merge func - this is used in the deployer here for merging default (gwclass-level) gwp with gateway-level gwp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Updated

@jenshu
Copy link
Contributor

jenshu commented Jan 13, 2025

@davidjumani are you still working on this or can it be closed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
keep pr updated signals bulldozer to keep pr up to date with base branch work in progress signals bulldozer to keep pr open (don't auto-merge)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants