Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement gcs state bucket parameter passing #377

Merged
merged 11 commits into from
Feb 27, 2025
Merged
11 changes: 5 additions & 6 deletions .github/workflows/build_tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,10 @@ on:
tpu-type:
description: 'TPU Type'
required: true
default: 'v5p-8'
default: 'v4-8'
type: choice
options:
- v5p-8
- v5litepod-8
- v4-8
push:
branches: ["main","develop"]
pull_request: # By default this runs for types assigned, opened and synchronize.
Expand Down Expand Up @@ -67,15 +66,15 @@ jobs:
- name: set zone
id: set-zone
run: |
echo zone=europe-west4-b >> $GITHUB_OUTPUT
echo zone=us-central2-b >> $GITHUB_OUTPUT
- name: set tpu-type
id: set-tpu-type
run: |
echo tpu-type=v5p-8 >> $GITHUB_OUTPUT
echo tpu-type=v4-8 >> $GITHUB_OUTPUT
- name: set location
id: set-location
run: |
echo location=europe-west4 >> $GITHUB_OUTPUT
echo location=us-central2 >> $GITHUB_OUTPUT
install-dependencies:
needs: [set-variables]
runs-on: ubuntu-22.04
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cluster_create.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,4 +79,4 @@ jobs:
- name: Check xpk installation
run: xpk --help
- name: Create a Pathways-enabled XPK Cluster with 2x ${{inputs.tpu-type}} nodepools. Larger num-nodes to avoid master resizing.
run: python xpk.py cluster create-pathways --cluster ${{inputs.cluster-name}} --tpu-type=${{inputs.tpu-type}} --num-slices=1 --zone=${{inputs.zone}} --default-pool-cpu-machine-type=n1-standard-16 --default-pool-cpu-num-nodes=4 --reservation='${{ secrets.GCP_TPU_V5_RESERVATION }}' --enable-gcpfilestore-csi-driver --enable-gcsfuse-csi-driver --custom-cluster-arguments="${CLUSTER_ARGUMENTS}"
run: python xpk.py cluster create-pathways --cluster ${{inputs.cluster-name}} --tpu-type=${{inputs.tpu-type}} --num-slices=1 --zone=${{inputs.zone}} --default-pool-cpu-machine-type=n1-standard-16 --default-pool-cpu-num-nodes=4 --reservation='${{ secrets.GCP_TPU_V4_RESERVATION }}' --enable-gcpfilestore-csi-driver --enable-gcsfuse-csi-driver --custom-cluster-arguments="${CLUSTER_ARGUMENTS}"
2 changes: 1 addition & 1 deletion .github/workflows/cluster_private.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ jobs:
- name: Check xpk installation
run: xpk --help
- name: Create a Pathways-enabled private XPK Cluster with 2x ${{inputs.tpu-type}} nodepools. Larger num-nodes to avoid master resizing.
run: python xpk.py cluster create-pathways --cluster ${{inputs.cluster-name}}-private --private --tpu-type=${{inputs.tpu-type}} --num-slices=1 --zone=${{inputs.zone}} --default-pool-cpu-machine-type=n1-standard-16 --default-pool-cpu-num-nodes=4 --reservation='${{ secrets.GCP_TPU_V5_RESERVATION }}' --custom-cluster-arguments="${CLUSTER_ARGUMENTS}"
run: python xpk.py cluster create-pathways --cluster ${{inputs.cluster-name}}-private --private --tpu-type=${{inputs.tpu-type}} --num-slices=1 --zone=${{inputs.zone}} --default-pool-cpu-machine-type=n1-standard-16 --default-pool-cpu-num-nodes=4 --reservation='${{ secrets.GCP_TPU_V4_RESERVATION }}' --custom-cluster-arguments="${CLUSTER_ARGUMENTS}"
- name: Verify the created cluster is private
run: gcloud container clusters describe ${{inputs.cluster-name}}-private --location=${{inputs.location}} --format="value(privateClusterConfig.enablePrivateNodes)" | grep 'True' || (echo 'The created cluster is not private.' && exit 1)
- name: Delete the cluster created
Expand Down
Loading