Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Introduce shared GitHub Actions #1451

Merged
merged 57 commits into from
Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
6bd2502
chore: Introduce shared GitHub Actions
c-pius Apr 5, 2024
4add18d
added workflow
c-pius Apr 5, 2024
0fbb9d9
fix path
c-pius Apr 5, 2024
9be041d
clean test
c-pius Apr 5, 2024
20a4976
wait-for-image-build action
c-pius Apr 5, 2024
0d2f2b6
local path
c-pius Apr 5, 2024
1712390
checkout
c-pius Apr 5, 2024
4ec638a
introduce install kubectl
c-pius Apr 8, 2024
73c3dcf
delete hello world
c-pius Apr 8, 2024
9b3d2d1
use minor k8s version
c-pius Apr 8, 2024
ab8cfdb
temp speedup image build
c-pius Apr 8, 2024
ee46675
parsing k8s version
c-pius Apr 8, 2024
b86d13d
introduce install istioctl
c-pius Apr 8, 2024
131e05e
introduce install kyma-cli
c-pius Apr 8, 2024
2f814ba
introduce install cmctl
c-pius Apr 8, 2024
1abc38a
introduce install k3d
c-pius Apr 8, 2024
ce3aaa6
use setup-go
c-pius Apr 8, 2024
25e635e
checkout lifecycle-manager to path
c-pius Apr 8, 2024
2681c6b
complete setup-tools
c-pius Apr 8, 2024
e3c4943
introduce configure-hosts and setup-test-clusters
c-pius Apr 8, 2024
c5c6e4b
introduce create-k3d-cluster
c-pius Apr 8, 2024
c949d05
introduce create kcp cluster
c-pius Apr 9, 2024
a02f9ab
use explicit k8s version input
c-pius Apr 9, 2024
d1bd7ee
introduce configure kubectl
c-pius Apr 9, 2024
210129c
add debugging comment
c-pius Apr 9, 2024
9de88fb
remove quotes
c-pius Apr 9, 2024
ccb562c
fix k3d install command
c-pius Apr 9, 2024
f577dd4
rename command
c-pius Apr 9, 2024
8dc5a6d
introduce deploy istio
c-pius Apr 9, 2024
48f1ebe
introduce deploy cert manager
c-pius Apr 9, 2024
8c58462
test shared configuration
c-pius Apr 9, 2024
e18dfcf
define variables in step
c-pius Apr 9, 2024
becbdec
try fixing istio version
c-pius Apr 9, 2024
c6a4b6d
add klm veresion tag in image repo configuration
c-pius Apr 9, 2024
04367c8
make k8s version configurable
c-pius Apr 9, 2024
4d6ea9a
add matrix
c-pius Apr 9, 2024
3a38e3e
add patch purge finalizer flags
c-pius Apr 9, 2024
0f3b62c
complete patches
c-pius Apr 9, 2024
9e3c4df
rename step
c-pius Apr 9, 2024
66b18d8
add checkout template-operator
c-pius Apr 9, 2024
90ac58a
add deploy template operator
c-pius Apr 9, 2024
6547373
add expose metrics endpiont
c-pius Apr 9, 2024
1e9f894
re-enable image build; add execution of e2e tests
c-pius Apr 9, 2024
d28e100
add names
c-pius Apr 9, 2024
5b662db
remove obsolete image build status name
c-pius Apr 9, 2024
11a37d2
pretifying
c-pius Apr 9, 2024
2a0c3c7
re-add workflow dispatch trigger
c-pius Apr 9, 2024
700df8f
outsorce gosumdb to get configuratin
c-pius Apr 9, 2024
c2b38b6
split deploy lifecycle-manager
c-pius Apr 9, 2024
66be3bd
pretifying
c-pius Apr 10, 2024
2445fb0
remove unnecessary kubectl context setting
c-pius Apr 10, 2024
1bd1178
add new rbac-privileges test
c-pius Apr 10, 2024
fe1fce6
fix typo
c-pius Apr 10, 2024
a3a9cec
merged new test file into old one
c-pius Apr 10, 2024
0512e0b
reset name
c-pius Apr 10, 2024
85683e5
Merge branch 'main' into chore/shared-github-actions
lindnerby Apr 12, 2024
b95c65f
Merge branch 'main' into chore/shared-github-actions
jeremyharisch Apr 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/actions/configure-hosts/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Configure hosts
description: Introduces an entry to the /etc/hosts file resolving k3d-kcp-registry to localhost.
runs:
using: composite
steps:
- name: Configure hosts
shell: bash
run: |
FILE=/etc/hosts
if [ -f "$FILE" ]; then
sudo echo "127.0.0.1 k3d-kcp-registry" | sudo tee -a $FILE
else
echo "$FILE does not exist."
exit 1
fi
echo "/etc/hosts file patched"
32 changes: 32 additions & 0 deletions .github/actions/create-k3d-cluster/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Create k3d cluster
description: Creates a k3d cluster.
inputs:
cluster_name:
description: The name of the cluster to create.
required: true
k8s_version:
description: The version of k8s to use.
required: true
args:
description: Additional arguments to pass to the k3d cluster create command separated by semicolon (;).
required: false
runs:
using: composite
steps:
- name: Create k3d cluster
shell: bash
run: |
IFS=';' read -ra splitted_args <<< "${{ inputs.args }}"

create_command="k3d cluster create ${{ inputs.cluster_name }} "

for arg in "${splitted_args[@]}";
do
create_command+="$arg "
done

create_command+="--image rancher/k3s:v${{ inputs.k8s_version }}-k3s1 "
create_command+="--k3s-arg --disable=traefik@server:* "
create_command+="--k3s-arg --tls-san=host.k3d.internal@server:* "

eval ${create_command}
14 changes: 14 additions & 0 deletions .github/actions/deploy-cert-manager/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Deploy cert-manager
description: Deploys cert-manager in the currently configured context.
inputs:
cert_manager_version:
description: The version of cert-manager to deploy. For example, 1.13.3.
required: true
runs:
using: composite
steps:
- name: Deploy cert-manager
shell: bash
run: |
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v${{ inputs.cert_manager_version }}/cert-manager.yaml
cmctl check api --wait=2m
9 changes: 9 additions & 0 deletions .github/actions/deploy-istio/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
name: Deploy istio
description: Deploys istio in the currently configured context.
runs:
using: composite
steps:
- name: Deploy istio
shell: bash
run: |
istioctl install --set profile=demo -y
91 changes: 91 additions & 0 deletions .github/actions/deploy-lifecycle-manager-e2e/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Deploy lifecycle-manager E2E
description: Applies E2E test-specific patches to the lifecycle-manager kustomize and deploys it.
inputs:
klm_version_tag:
description: The version tag for the KLM image. For example, PR-123.
required: true
klm_image_repo:
description: The repository for the KLM image. For example, dev.
required: true
runs:
using: composite
steps:
- name: Patch purge finalizer flags
if: ${{ matrix.e2e-test == 'purge-controller' || matrix.e2e-test == 'purge-metrics'}}
working-directory: lifecycle-manager
shell: bash
run: |
pushd config/watcher_local_test
echo \
"- op: add
path: /spec/template/spec/containers/0/args/-
value: --enable-purge-finalizer=true
- op: add
path: /spec/template/spec/containers/0/args/-
value: --purge-finalizer-timeout=5s" >> purge_finalizer.yaml
cat purge_finalizer.yaml
kustomize edit add patch --path purge_finalizer.yaml --kind Deployment
popd
- name: Patch metrics cleanup interval
if : ${{ matrix.e2e-test == 'kyma-metrics' }}
working-directory: lifecycle-manager
shell: bash
run: |
pushd config/watcher_local_test
echo \
"- op: add
path: /spec/template/spec/containers/0/args/-
value: --metrics-cleanup-interval=1" >> metrics_cleanup.yaml
cat metrics_cleanup.yaml
kustomize edit add patch --path metrics_cleanup.yaml --kind Deployment
popd
- name: Patch self signed certificate lifetime
if: ${{matrix.e2e-test == 'self-signed-certificate-rotation'}}
working-directory: lifecycle-manager
shell: bash
run: |
pushd config/watcher_local_test
echo \
"- op: add
path: /spec/template/spec/containers/0/args/-
value: --self-signed-cert-duration=1h
- op: add
path: /spec/template/spec/containers/0/args/-
value: --self-signed-cert-renew-before=59m
- op: add
path: /spec/template/spec/containers/0/args/-
value: --self-signed-cert-renew-buffer=1m" >> self-signed-cert.yaml
cat self-signed-cert.yaml
kustomize edit add patch --path self-signed-cert.yaml --kind Deployment
popd
- name: Patch CA certificate renewBefore
if: ${{matrix.e2e-test == 'ca-certificate-rotation'}}
working-directory: lifecycle-manager
shell: bash
run: |
pushd config/watcher_local_test
echo \
"- op: replace
path: /spec/renewBefore
value: 59m
- op: replace
path: /spec/duration
value: 1h">> certificate_renewal.yaml
cat certificate_renewal.yaml
kustomize edit add patch --path certificate_renewal.yaml --kind Certificate --group cert-manager.io --version v1 --name watcher-serving-cert
popd
- name: Deploy LM local testing kustomize
uses: ./lifecycle-manager/.github/actions/deploy-lifecycle-manager
with:
klm_version_tag: ${{ inputs.klm_version_tag }}
klm_image_repo: ${{ inputs.klm_image_repo }}
- name: Expose Metrics Endpoint
working-directory: lifecycle-manager
if: ${{ matrix.e2e-test == 'kyma-metrics' ||
matrix.e2e-test == 'purge-metrics' ||
matrix.e2e-test == 'self-signed-certificate-rotation' ||
matrix.e2e-test == 'mandatory-module-metrics'
}}
shell: bash
run: |
kubectl patch svc klm-metrics-service -p '{"spec": {"type": "LoadBalancer"}}' -n kcp-system
31 changes: 31 additions & 0 deletions .github/actions/deploy-lifecycle-manager/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Deploy lifecycle-manager
description: Deploys lifecycle-manager.
inputs:
klm_version_tag:
description: The version tag for the KLM image. For example, PR-123.
required: true
klm_image_repo:
description: The repository for the KLM image. For example, dev.
required: true
runs:
using: composite
steps:
- name: Deploy LM local testing kustomize
working-directory: lifecycle-manager
shell: bash
run: |
maxRetry=5
for retry in $(seq 1 $maxRetry)
do
if make local-deploy-with-watcher IMG=europe-docker.pkg.dev/kyma-project/${{ inputs.klm_image_repo }}/lifecycle-manager:${{ inputs.klm_version_tag }}; then
kubectl wait pods -n kcp-system -l app.kubernetes.io/name=lifecycle-manager --for condition=Ready --timeout=90s
jeremyharisch marked this conversation as resolved.
Show resolved Hide resolved
echo "KLM deployed successfully"
exit 0
elif [[ $retry -lt $maxRetry ]]; then
echo "Deploy encountered some error, will retry after 20 seconds"
sleep 20
else
echo "KLM deployment failed"
exit 1
fi
done
90 changes: 90 additions & 0 deletions .github/actions/deploy-template-operator/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
name: Deploy template-operator
description: Deploys a test-specific template-operator.
runs:
using: composite
steps:
- name: Create Template Operator Module and apply
working-directory: template-operator
if: ${{ matrix.e2e-test == 'kyma-metrics' ||
matrix.e2e-test == 'non-blocking-deletion' ||
matrix.e2e-test == 'purge-controller' ||
matrix.e2e-test == 'purge-metrics' ||
matrix.e2e-test == 'kyma-deprovision-with-foreground-propagation' ||
matrix.e2e-test == 'kyma-deprovision-with-background-propagation' ||
matrix.e2e-test == 'module-consistency' ||
matrix.e2e-test == 'skip-manifest-reconciliation' ||
matrix.e2e-test == 'misconfigured-kyma-secret'
}}
shell: bash
run: |
make build-manifests
kyma alpha create module --module-config-file ./module-config.yaml --path . --registry k3d-kcp-registry:5111 --insecure
sed -i 's/k3d-kcp-registry:5111/k3d-kcp-registry:5000/g' ./template.yaml
kubectl get crds
kubectl apply -f template.yaml
- name: Create Template Operator Module for regular and fast channels
working-directory: lifecycle-manager
if: ${{ matrix.e2e-test == 'module-upgrade-channel-switch' ||
matrix.e2e-test == 'module-upgrade-new-version' ||
matrix.e2e-test == 'upgrade-under-deletion'
}}
shell: bash
run: |
kubectl apply -f tests/moduletemplates/moduletemplate_template_operator_v2_fast.yaml
kubectl apply -f tests/moduletemplates/moduletemplate_template_operator_v1_regular.yaml
- name: Create Template Operator Module as Mandatory Module
working-directory: lifecycle-manager
if: ${{ matrix.e2e-test == 'mandatory-module' ||
matrix.e2e-test == 'mandatory-module-metrics'
}}
shell: bash
run: |
kubectl apply -f tests/moduletemplates/mandatory_moduletemplate_template_operator_v1.yaml
- name: Apply Template Operator Module V2, fast channel
working-directory: ./lifecycle-manager
if: ${{ matrix.e2e-test == 'non-blocking-deletion' }}
shell: bash
run: |
kubectl apply -f tests/moduletemplates/moduletemplate_template_operator_v2_fast.yaml
- name: Create Template Operator Module with final state and final deletion state as `Warning` and apply
working-directory: template-operator
if: ${{ matrix.e2e-test == 'module-status-propagation'}}
shell: bash
run: |
pushd config/default
echo \
"- op: replace
path: /spec/template/spec/containers/0/args/1
value: --final-state=Warning
- op: replace
path: /spec/template/spec/containers/0/args/2
value: --final-deletion-state=Warning" >> warning_patch.yaml
cat warning_patch.yaml
kustomize edit add patch --path warning_patch.yaml --kind Deployment
popd
kyma alpha create module --kubebuilder-project --channel=regular --name kyma.project.io/module/template-operator --version 1.1.1 --path . --registry k3d-kcp-registry:5111 --insecure --module-archive-version-overwrite /
sed -i 's/k3d-kcp-registry:5111/k3d-kcp-registry:5000/g' ./template.yaml
kubectl get crds
kubectl apply -f template.yaml
- name: Create Template Operator Module without default CR and apply
working-directory: template-operator
if: ${{ matrix.e2e-test == 'module-without-default-cr' }}
shell: bash
run: |
make build-manifests
echo "name: kyma-project.io/module/template-operator
channel: regular
version: v1.0.0
manifest: template-operator.yaml
security: sec-scanners-config.yaml
annotations:
operator.kyma-project.io/doc-url: https://kyma-project.io" >> module-config-no-cr.yaml
kyma alpha create module \
--module-config-file ./module-config-no-cr.yaml \
--path . \
--registry k3d-kcp-registry:5111 \
--insecure
sed -i 's/k3d-kcp-registry:5111/k3d-kcp-registry:5000/g' ./template.yaml
kubectl get crds
kubectl apply -f template.yaml

15 changes: 15 additions & 0 deletions .github/actions/export-kubeconfigs/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Export kubeconfigs
description: Merges the configs from KCP and SKR k3d clusters into the default kubeconfig and exports the same as environment variables KCP_KUBECONFIG and SKR_KUBECONFIG.
inputs:
context_name:
description: The name of the context to use.
required: true
runs:
using: composite
steps:
- name: Export kubeconfigs
shell: bash
run: |
k3d kubeconfig merge -a -d
echo "KCP_KUBECONFIG=$(k3d kubeconfig write kcp)" >> $GITHUB_ENV
echo "SKR_KUBECONFIG=$(k3d kubeconfig write skr)" >> $GITHUB_ENV
38 changes: 38 additions & 0 deletions .github/actions/get-configuration/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Get configuration
description: Defines configuration variables such as versions. Exposes globally needed environment variables.
outputs:
k8s_version:
description: The version of k8s to use. For example, 1.28.7.
value: ${{ steps.define-variables.outputs.k8s_version }}
istio_version:
description: The version of Istio to install. For example, 1.20.3.
value: ${{ steps.define-variables.outputs.istio_version }}
k3d_version:
description: The version of k3d to install. For example, 5.6.0.
value: ${{ steps.define-variables.outputs.k3d_version }}
cert_manager_version:
description: The version of cert-manager to deploy. For example, 1.13.3.
value: ${{ steps.define-variables.outputs.cert_manager_version }}
klm_version_tag:
description: The version tag for the KLM image. For example, PR-123.
value: ${{ steps.define-variables.outputs.klm_version_tag }}
klm_image_repo:
description: The repository for the KLM image. For example, dev.
value: ${{ steps.define-variables.outputs.klm_image_repo }}
runs:
using: composite
steps:
- name: Define variables
id: define-variables
shell: bash
run: |
echo "k8s_version=${{ github.event.inputs.k8s_version || '1.28.7' }}" >> $GITHUB_OUTPUT
echo "istio_version=1.20.3" >> $GITHUB_OUTPUT
echo "k3d_version=5.6.0" >> $GITHUB_OUTPUT
echo "cert_manager_version=1.13.3" >> $GITHUB_OUTPUT
echo "klm_version_tag=PR-${{ github.event.pull_request.number }}" >> $GITHUB_OUTPUT
echo "klm_image_repo=dev" >> $GITHUB_OUTPUT
Comment on lines +33 to +34
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment: these here used to be a conditional based on whether the job is running for a main build or a PR build. Since we removed the post main build, I removed this conditional now.

- name: Expose environment variables
shell: bash
run: |
echo "GOSUMDB=off" >> $GITHUB_ENV
Comment on lines +24 to +38
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment: I opted to have everything just needed in the test setup as "regular" vars, and to only export those things as env vars which are needed downstream by the tests.

14 changes: 14 additions & 0 deletions .github/actions/install-cmctl/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
name: Install cmctl
description: Downloads cmctl (cert-manager Command Line Tool) and installs it locally.
runs:
using: composite
steps:
- name: Install cmctl
shell: bash
run: |
OS=$(go env GOOS)
ARCH=$(go env GOARCH)

curl -fsSL -o cmctl.tar.gz https://github.com/cert-manager/cert-manager/releases/latest/download/cmctl-$OS-$ARCH.tar.gz
tar xzf cmctl.tar.gz
sudo mv cmctl /usr/local/bin
15 changes: 15 additions & 0 deletions .github/actions/install-istioctl/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Install istioctl
description: Downloads istioctl and installs it locally.
inputs:
istio_version:
description: The version of istioctl to install. For example, 1.20.3.
required: true
runs:
using: composite
steps:
- name: Install istioctl
shell: bash
run: |
curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${{ inputs.istio_version }} TARGET_ARCH=x86_64 sh -
chmod +x istio-${{ inputs.istio_version }}/bin/istioctl
mv istio-${{ inputs.istio_version }}/bin/istioctl /usr/local/bin
13 changes: 13 additions & 0 deletions .github/actions/install-k3d/action.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Install k3d
description: Downloads k3d and installs it locally.
inputs:
k3d_version:
description: The version of k3d to install. For example, 5.6.0.
required: true
runs:
using: composite
steps:
- name: Install k3d
shell: bash
run: |
wget -qO - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | TAG=v${{ inputs.k3d_version }} bash
Loading
Loading