Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[idea] native support for multiple clusters #180

Open
jan-janssen opened this issue May 4, 2023 · 0 comments
Open

[idea] native support for multiple clusters #180

jan-janssen opened this issue May 4, 2023 · 0 comments

Comments

@jan-janssen
Copy link
Member

Currently, switching between clusters is a manual task:

qa = QueueAdapter()
qa.list_clusters()
>>> 'cluster_1', 'cluster_2'
qa.switch_cluster(cluster_name='cluster_1')
qa.submit_job(...)

It would be great if pysqa could switch between the clusters seamlessly. Basically by selecting the queue pysqa could automatically identify which cluster is used so that the different cluster behave like one to the user.

There are a couple of challenges:

  • Two clusters could potentially return the same queue ID. The current approach is to add an integer at the end of the queue ID which limits the number of clusters to 10.
  • The other issue is that getting the status of all clusters increases linearly with the number of clusters. This could be executed in parallel, still that would only be more complex.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant