Orphaned shard files can be left behind by retention service (Backport to 1.11) #24543
Comments
gwossum
added a commit
that referenced
this issue
Jan 4, 2024
Under certain circumstances, the retention service can fail to delete shards from the store in a timely manner. When the shard groups are pruned based on age, this leaves orphaned shard files on the disk. The retention service will then not attempt to remove the obsolete shard files because the meta store does not know about them. This can cause excessive disk space usage for some users. This corrects that by requiring shards files be deleted before they can be removed from the meta store. Backport via clean cherry-pick of #24530. fixes: #24543 (cherry picked from commit 7bd3f89)
gwossum
added a commit
that referenced
this issue
Jan 4, 2024
Under certain circumstances, the retention service can fail to delete shards from the store in a timely manner. When the shard groups are pruned based on age, this leaves orphaned shard files on the disk. The retention service will then not attempt to remove the obsolete shard files because the meta store does not know about them. This can cause excessive disk space usage for some users. This corrects that by requiring shards files be deleted before they can be removed from the meta store. Backport via clean cherry-pick of #24530. fixes: #24543 (cherry picked from commit 7bd3f89)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Backport #24530 to 1.11
The text was updated successfully, but these errors were encountered: