Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transport.compress and transport.extract #6740

Open
khsrali opened this issue Jan 30, 2025 · 0 comments
Open

transport.compress and transport.extract #6740

khsrali opened this issue Jan 30, 2025 · 0 comments
Labels
type/feature request status undecided

Comments

@khsrali
Copy link
Contributor

khsrali commented Jan 30, 2025

We plan to develop more types for StashMode such as zip or tar, etc.
Also potentially an unstashing feature that could "undo" whatever stashing did.

For that we need to develop function execmanager.py::stash_calculation to compress a list of paths starting from a common root.
Once can simply use transport.exec_command_wait, however aiida-firecrest does not and will not support that.

So now that we know that, and that we are in early stage of developing this feature, I suggest we keep that in mind, and introduce features like transport.compress and transport.extract.
So that the action of compressing and extracting is left to the hands of the transport plugin. It's up to transport plugin how to handle that, either by executing a command or sending a REST request.

In my opinion this makes a lot of sense. Also in general, we may need to have the codebase interface as independent as possible from transport.exec_command_wait.
So execmanager.py::stash_calculation would only call on transport.compress and transport.extract without being bothered by how is that done.

discussed with @agoscinski

@khsrali khsrali added the type/feature request status undecided label Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature request status undecided
Projects
None yet
Development

No branches or pull requests

1 participant