feat(mobile): add deduplication hash to backup headers #16133
+118
−42
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rom being reuploaded.
bo0tzz
changed the title
bo0tzz
changed the title
added 3 commits
February 16, 2025 16:10
Backup service now, instead of loading all local assets, is getting the current asset from the DB using the local ID.
|
Hello, and thank you for the PR. I ran some tests and noticed the feature isn't working as intended. Perhaps it is related to our Dart The duplicated assets are still getting uploaded to the server. You can reproduce by connecting to a slower WIFI frequency (2.4GHz) and uploading a larger duplicated file, the expected behavior is that the request would be skipped, but I am still seeing the upload bar progressing |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR introduces a mechanism to prevent duplicated asset uploads to the server. It leverages the existing x-immich-checksum header to send the asset’s SHA-1 checksum for validation before the full upload takes place.
Additionally, this PR implements a single-file checksum bridge between iOS and Android for Flutter, ensuring consistency across platforms.
By preventing unnecessary uploads, especially of large files, this change optimizes bandwidth usage and improves overall efficiency.
Next Steps
Since images with different resolutions or compression settings generate distinct SHA-1 checksums, the next step will be to explore a more robust approach to image deduplication.
Fixes
#1553 - Move to client side hashing
How Has This Been Tested?
I ran the iOS and Android app, tested the backup with new images, tested uploading an image to the server through the web, and then added the same asset to the device. I tried to back up this new file, and it detected the duplicate.
Checklist:
src/servicesuses repositories implementations for database calls, filesystem operations, etc.src/repositories/is pretty basic/simple and does not have any immich specific logic (that belongs insrc/services)