Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

callbacks: wrap non-dvc callbacks passed via fsspec #227

Merged
merged 2 commits into from
Nov 10, 2023

Conversation

pmrowla
Copy link
Contributor

@pmrowla pmrowla commented Nov 2, 2023

@pmrowla pmrowla requested a review from skshetry November 2, 2023 08:44
return maybe_callback
if isinstance(maybe_callback, Callback):
return maybe_callback
return _FsspecCallbackWrapper(maybe_callback)

@classmethod
def as_tqdm_callback(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we move as_tqdm_callback and as_callback to the mixin as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could move as_tqdm_callback, but I think as_callback should stay here to override the default fsspec implementation

@skshetry
Copy link
Member

skshetry commented Nov 7, 2023

Sorry for the delay. The code changes look good to me.

Alternatively, we could also try wrapping in between dvc_objects.fs.Filesystem <> fsspec.AbstractFileSystem, but that might end up being more complicated.

@codecov-commenter
Copy link

codecov-commenter commented Nov 7, 2023

Codecov Report

Attention: 9 lines in your changes are missing coverage. Please review.

Files Coverage Δ
src/dvc_objects/fs/local.py 60.15% <100.00%> (ø)
tests/fs/test_callbacks.py 100.00% <100.00%> (ø)
src/dvc_objects/fs/generic.py 26.56% <0.00%> (ø)
src/dvc_objects/fs/callbacks.py 79.09% <73.33%> (-1.81%) ⬇️

📢 Thoughts on this report? Let us know!

@pmrowla pmrowla marked this pull request as ready for review November 10, 2023 10:24
@pmrowla pmrowla merged commit bff4c06 into iterative:main Nov 10, 2023
@pmrowla pmrowla deleted the callbacks-fsspec branch November 10, 2023 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dvcfs: Can't copyfile when fsspec passes a non-DVC callback
3 participants