Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USBStorageDriver cleanup remote resource #1588

Closed

Conversation

esben
Copy link
Contributor

@esben esben commented Jan 30, 2025

Description

Change the USBStorageDriver class to cleanup remote files (ManagedFile) when the remote copy of the files have been used.
This helps avoid running out of disk space on the remote resource, especially if using USBStorageDriver with large image files.

Checklist

  • Documentation for the feature
  • Tests for the feature
  • The arguments and description in doc/configuration.rst have been updated
  • Add a section on how to use the feature to doc/usage.rst
  • Add a section on how to use the feature to doc/development.rst
  • PR has been tested (tested in our local setup, verifying that no files were left after running tests that without this change left large image files in /var/cache/labgrid) (tested with a rebase to stable-24.0 branch, as we don't have a working test setup on master branch)
  • Man pages have been regenerated

esben added 2 commits January 30, 2025 14:29
When using ManagedFile with remote resources without NFS, files copied
to the remote by sync_to_resource() method will not be deleted
automatically.

While it is of-course possible to use something like systemd-tmpfiles
to cleanup after a while, files might pile up and cause out of diskspace
situations before systemd-tmpfiles gets to it.

This method can be used by users of ManagedFile objects to cleanup the
remote copy of the file when it is no longer needed.

Signed-off-by: Esben Haabendal <[email protected]>
Delete the remote resource copy of the files being written to USB storage
when done, avoiding filling up /var/cache/labgrid over time.

Keep in mind, that this this can be used for writing quite large
files, so can fill up storage on an exporter quite fast.

Signed-off-by: Esben Haabendal <[email protected]>
Copy link

codecov bot commented Jan 30, 2025

Codecov Report

Attention: Patch coverage is 5.88235% with 16 lines in your changes missing coverage. Please review.

Project coverage is 55.9%. Comparing base (f81f479) to head (4aba188).
Report is 6 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
labgrid/util/managedfile.py 8.3% 11 Missing ⚠️
labgrid/driver/usbstoragedriver.py 0.0% 5 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master   #1588     +/-   ##
========================================
- Coverage    56.0%   55.9%   -0.1%     
========================================
  Files         170     170             
  Lines       13307   13322     +15     
========================================
+ Hits         7458    7459      +1     
- Misses       5849    5863     +14     
Flag Coverage Δ
3.10 55.9% <5.8%> (-0.1%) ⬇️
3.11 55.9% <5.8%> (-0.1%) ⬇️
3.12 55.9% <5.8%> (-0.1%) ⬇️
3.13 55.9% <5.8%> (-0.1%) ⬇️
3.9 56.0% <5.8%> (-0.1%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Emantor Emantor self-assigned this Feb 1, 2025
@Emantor
Copy link
Member

Emantor commented Feb 4, 2025

Please see the "Attention" box in the Exporter documentation. This PR disables the caching completely, you should be able to configure the cleanup interval for your infrastructure to shorter intervals if storage is tight.

@Emantor Emantor closed this Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants