Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove usage of *_STORAGE_API_ENDPOINT` environment variables for providers #856

Merged

Conversation

shreyas-s-rao
Copy link
Contributor

How to categorize this PR?

/area security
/kind task

What this PR does / why we need it:
Remove usage of *_STORAGE_API_ENDPOINT` environment variables for providers, since credentials from secrets (including API endpoints) should not be passed as environment variables to the backup-restore container.

Which issue(s) this PR fixes:
Fixes gardener/etcd-backup-restore#727

Special notes for your reviewer:
/block
To be merged only after gardener/etcd-backup-restore#759 is merged, released and etcdbr version v0.30.0 is updated in druid.
For local testing purpose, I have included the PR build image from https://concourse.ci.gardener.cloud/teams/gardener/pipelines/etcd-backup-restore-master/jobs/master-pull-request-job/builds/37. This commit will be removed before merging.

@renormalize please let me know if we can pass domain to Azurite (with format http(s)://<storageAccount>.<domain>) instead of the entire URL.

Release note:

Remove usage of *_STORAGE_API_ENDPOINT` environment variables for Google and Azure providers. Storage API endpoint / domain will instead be directly consumed by etcd-backup-restore from the mounted backup secret.

@shreyas-s-rao shreyas-s-rao added this to the v0.23.0 milestone Aug 12, 2024
@shreyas-s-rao shreyas-s-rao requested a review from a team as a code owner August 12, 2024 08:27
@gardener-robot gardener-robot added needs/review Needs review area/security Security related kind/task General task size/s Size of pull request is small (see gardener-robot robot/bots/size.py) labels Aug 12, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 12, 2024
@shreyas-s-rao
Copy link
Contributor Author

/assign @renormalize @unmarshall

@gardener-robot-ci-1 gardener-robot-ci-1 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Aug 12, 2024
renormalize

This comment was marked as outdated.

@gardener-robot gardener-robot added the needs/changes Needs (more) changes label Aug 12, 2024
@renormalize
Copy link
Member

As discussed off GitHub with @shreyas-s-rao, passing domain in the http(s)://<storageAccount>.<domain> format is supported by Azurite.

The way to perform this is to add:

127.0.0.1 account1.blob.localhost

to the hosts file of the host machine where etcd-backup-restore tests are running so that https://<storage-account>.blob.localhost resolves to the Azurite process. One simply needs to pass the storage account name that is configured to be used for Azurite, and the domain as blob.localhost.

However, it has to be kept in mind to not pass the disableProductStyleUrl flag to Azurite at startup. More can be found here.

Doing this will break the current way the Azurite make targets work in etcd-druid, so for a few commits Azurite will not be functional with etcd-druid for local development and testing.

@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Aug 14, 2024
@shreyas-s-rao
Copy link
Contributor Author

Added this as a point in #782

@shreyas-s-rao
Copy link
Contributor Author

/hold until gardener/etcd-backup-restore#759 is merged, released and etcdbr version v0.30.0 is updated in druid.

@gardener-robot gardener-robot added the reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies label Aug 14, 2024
@shreyas-s-rao shreyas-s-rao self-assigned this Aug 14, 2024
@ashwani2k
Copy link
Collaborator

/unhold as gardener/etcd-backup-restore#759 is merged.
@shreyas-s-rao Shall we merge it?

@gardener-robot gardener-robot removed the reviewed/do-not-merge Has no approval for merging as it may break things, be of poor quality or have (ext.) dependencies label Aug 23, 2024
@shreyas-s-rao shreyas-s-rao force-pushed the task/remove-storage-endpoint-env-var branch from 2ac9b10 to 85f287d Compare August 23, 2024 08:35
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 23, 2024
@shreyas-s-rao
Copy link
Contributor Author

@ashwani2k I've removed the temp commit now, so the PR can be merged once the tests pass.

@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 23, 2024
@shreyas-s-rao
Copy link
Contributor Author

/test pull-etcd-druid-e2e-kind

@shreyas-s-rao
Copy link
Contributor Author

@ashwani2k can you please raise a PR to update etcdbr image to v0.30.0 in images.yaml, along with the relevant release notes? I'm surprised that the PR was not raised automatically.

@shreyas-s-rao shreyas-s-rao force-pushed the task/remove-storage-endpoint-env-var branch from 85f287d to 0ca1ca1 Compare August 23, 2024 09:20
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 23, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Aug 23, 2024
@shreyas-s-rao
Copy link
Contributor Author

/test pull-etcd-druid-e2e-kind-nondistroless-etcd

@shreyas-s-rao shreyas-s-rao force-pushed the task/remove-storage-endpoint-env-var branch from 0ca1ca1 to dcbb59e Compare September 9, 2024 04:37
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Sep 9, 2024
@shreyas-s-rao
Copy link
Contributor Author

@ashwani2k I have rebased the PR, so it can now be merged, after the tests pass of course.

@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Sep 9, 2024
Copy link
Member

@renormalize renormalize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with the v0.30.1 of etcd-backup-restore, and all behavior is as expected. Thanks.

@ashwani2k ashwani2k merged commit d3ac301 into gardener:master Sep 9, 2024
11 checks passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Sep 9, 2024
@shreyas-s-rao shreyas-s-rao deleted the task/remove-storage-endpoint-env-var branch September 9, 2024 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/security Security related kind/task General task needs/changes Needs (more) changes needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/review Needs review size/s Size of pull request is small (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Do not store Kubernetes secrets as environment variables
8 participants