Advisory: git "unable to clone" or "repository corrupt" errors diagnosis

[ramses0]

I ran into a weird one and wanted to share:

https://stackoverflow.com/questions/4826639/repack-of-git-repository-fails

https://github.com/git/git/blob/master/Documentation/config/pack.txt

https://confluence.atlassian.com/bitbucketserverkb/git-operations-fail-due-to-git-pack-objects-died-with-error-849169949.html

...symptoms were: I was (suddenly) unable to sync via passforios with a bunch of "unable to clone", "underlying error", "credentials error, repository may be corrupt" (and similar)

When doing the diagnosis and troubleshooting I ran into the above articles and can explain or understand what happened a bit better.

Make sure git fsck, git gc, and git repack all work properly. In retrospect what happened was I host my .password-store.git repo at a cheap shared ssh-capable hosting provider. They likely migrated me from machine-with-24cpu to machine-with-128cpu, and then when git goes to do it's usual send-pack / receive-pack, it was getting smacked down by ulimit settings (as it's a shared host), and by default, git gc (nee: repack) was trying to spawn as many threads as there are processors on the server. In this case the host effectively saw a user process spawning 128 threads and was denying / killing the process which manifested itself as "corrupt repository" (since it couldn't properly fsck ; gc ; repack).

The answer was to effectively git config --local pack.threads 4 (or something) otherwise I was getting fatal: failed to run repack and similar which were very difficult to diagnose from within the iOS client. When doing it via pass git push ... terminal CLI, maybe there's better error-handling (or synergy, or client version mismatches or something) which wasn't triggering the issue.

In short: verify that git fsck ; git gc ; git repack all work correctly, and learn a little about git config pack.* if you're hitting some sorts of limits.

No real action is necessary on the passforios side unless it would make sense to have a troubleshooting section, better docs, or some sort of fit-for-purpose or acceptance-test that could be run against the repository, so feel free to close this issue. I mainly just wanted to report it so that it would potentially be found by future people searching for issues like this.

Thanks!