Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup cache dir at mount and exit #620

Merged

Conversation

dannycjones
Copy link
Contributor

Description of change

We want to clean the directory used for caching at mount time and exit time to keep usage of the disk under control, especially since we do not intend to support cache re-use (at least initially).

This change adds a new managed directory type, which will create the directory we expect and deleted it on drop, if close(mut self) was not already called. We introduce close since the drop implementation did not appear to always be invoked.

Relevant issues: #255

Does this change impact existing behavior?

No change to released features. Only impacts caching, currently behind a feature flag.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

}

/// Create an owned copy of the managed path
pub fn as_path_buf(&self) -> PathBuf {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems unnecessary. Can't you do something like .as_path().to_owned() if required?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can, but I found we were doing that a few times and it felt more ergonomic to just add the method here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left this in for now. We do use it at the moment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should remove. But not a blocker.

mountpoint-s3/src/main.rs Outdated Show resolved Hide resolved
mountpoint-s3/src/data_cache/disk_data_cache.rs Outdated Show resolved Hide resolved
mountpoint-s3/src/data_cache/cache_directory.rs Outdated Show resolved Hide resolved
@dannycjones dannycjones temporarily deployed to PR integration tests November 21, 2023 16:42 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests November 21, 2023 16:42 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests November 21, 2023 16:42 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests November 21, 2023 16:42 — with GitHub Actions Inactive
@dannycjones dannycjones requested a review from passaro November 21, 2023 16:45
Copy link
Contributor

@passaro passaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of nits. Entirely optional.

@@ -597,6 +601,14 @@ fn mount(args: CliArgs) -> anyhow::Result<FuseSession> {
fuse_config,
&bucket_description,
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could just ? here and simplify the rest

}

/// Create an owned copy of the managed path
pub fn as_path_buf(&self) -> PathBuf {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think we should remove. But not a blocker.

@dannycjones
Copy link
Contributor Author

Merging as is for now, happy to revisit some nits later.

@dannycjones dannycjones added this pull request to the merge queue Nov 21, 2023
Merged via the queue into awslabs:main with commit b1c1781 Nov 21, 2023
18 checks passed
@dannycjones dannycjones deleted the cleanup-cache-dir-at-mount-and-exit branch November 21, 2023 18:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants