Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Number of retries in the register_cluster() function should be more than 2 #1215

Open
jacekszlachtass opened this issue Jun 7, 2022 · 4 comments

Comments

@jacekszlachtass
Copy link

The number of retries in the register_cluster() function should be more than 2:

retry 2 run_command ${CMD}

I got this error twice in a row:

#asmcli: Registering the cluster as fow-sandbox-fow-sandbox-europe-west2...
#asmcli: [WARNING]: Failed, retrying...(1 of 2)
#asmcli: [WARNING]: Failed, retrying...(2 of 2)
#asmcli: [WARNING]: Command 'run_command gcloud container hub memberships register fow-sandbox-fow-sandbox-europe-west2 --project=fow-sandbox --enable-workload-identity --gke-uri=https://container.googleapis.com/v1/projects/fow-sandbox/locations/europe-west2/clusters/fow-sandbox' failed.

The solution was to manually run the command from the error message and re-run the asmcli command again.

@jacekszlachtass
Copy link
Author

Looks like the problem is not the number of retries, but the fact that it quickly fails. I tried with a higher number of retries, but without any luck. The only way around it is to manually run the "gcloud" command, which takes a while, and then re-run asmcli.

@jacekszlachtass
Copy link
Author

And the latest output:

asmcli: Verifying cluster registration.
asmcli: Registering the cluster as fow-sandbox-5udocewp...
asmcli: [WARNING]: Failed, retrying...(1 of 2)
asmcli: [WARNING]: Failed, retrying...(2 of 2)
asmcli: [WARNING]: Command 'run_command gcloud container hub memberships register fow-sandbox-5udocewp --project=fow-sandbox --enable-workload-identity --gke-uri=https://container.googleapis.com/v1/projects/fow-sandbox/locations/europe-west2/clusters/fow-sandbox' failed.
❯ gcloud container hub memberships register fow-sandbox-5udocewp --project=fow-sandbox --enable-workload-identity --gke-uri=https://container.googleapis.com/v1/projects/fow-sandbox/locations/europe-west2/clusters/fow-sandbox
kubeconfig entry generated for fow-sandbox.
Waiting for membership to be created...done.
Created a new membership [projects/fow-sandbox/locations/global/memberships/fow-sandbox-5udocewp] for the cluster [fow-sandbox-5udocewp]
Generating the Connect Agent manifest...
Deploying the Connect Agent on cluster [fow-sandbox-5udocewp] in namespace [gke-connect]...
Deployed the Connect Agent on cluster [fow-sandbox-5udocewp] in namespace [gke-connect].
Finished registering the cluster [fow-sandbox-5udocewp] with the Fleet.

@zerobfd
Copy link
Contributor

zerobfd commented Jun 7, 2022

Thanks for the report! Do you remember about how long the command took to run? I can adjust the timeout.

@jacekszlachtass
Copy link
Author

Hm, good question, something like 30-40 seconds. It is not quick, a timeout of 1 minute would probably make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants