Discrepancies on config file locking between Windows and Linux?

[darkfeline]

We have various tooling scripts which spawns many Git commands in parallel, including config setting commands.

We are seeing these errors specifically on Windows (and not Linux).

warning: unable to access 'C:/Users/blah/.gitconfig': Permission denied
fatal: unknown error occurred while reading the configuration files

I suspect this is due to the config setting Git commands racing and locking out other Git commands running simultaneously, and that is either due to a platform difference with Windows, or some modification Git for Windows has done to config locking semantics as part of adapting Git to Windows.

I am not that familiar with Windows unfortunately, so I would appreciate any insight into why this happens on Git for Windows and not Linux, and if anything can be done about it, e.g., by changing how Git for Windows opens/locks the config file to better match Linux semantics?

I am observing this behavior with 2.46.0, which I understand is not up to date. I don't have reason to suspect this is different with the latest version, but I apologize if it is. I will try to test with the latest but I need to deal with process overhead so it will take time.

(Also unfortunately, this is legacy code; I recognize that spawning many Git commands in parallel is not ideal, but it's hard to change)

[dscho]

The "Permission denied" error does not come from Git's internal file locking, at least not in an obvious way, because Git's strategy for locking files goes like this: Instead of writing to the file directly, it writes to a <filename>.lock file, then when that's done, it renames (atomically, overwriting) the .lock file to the actual file.

Most commonly the "Permission denied" error is shown when the file in question is opened by another process in an "exclusive" manner.

That process could be Git for Windows, trying to rename but failing because the original file exists and then trying to delete that original file first and re-trying the rename operation. It is a tiny time window, but of course it can be hit.

Another common scenario is where Microsoft Defender opens that file, however briefly that may be, and Git still is unable to read the config.

It is curious, though, that you are hitting this because such a racy scenario typically involves writing to the ~/.gitconfig file in a massively concurrent way. We hit this kind of issue quite often in the Scalar Functional Tests over at https://github.com/microsoft/git
where tons of tests exercise scalar concurrently, and all of them need to write to the "global" (i.e. user-wide) config; We worked around that problem via microsoft@8d3fe0f. But that only addresses file locking issues when writing concurrently, whereas your description clearly talks about reading.

It is of course possible that the issue might be addressed in the upcoming Git for Windows version which uses POSIX semantics when renaming, which might fix the underlying issue.

[darkfeline]

We do both reading and writing; some of the parallel Git commands are using git config to set config values and all/most of the commands read the config as part of normal operation.

It sounds similar to the problem you talked about with the Scalar Functional Tests, although the errors we're seeing appear to be reads failing due to a simultaneous config write, and not two simultaneous writes competing.

It is of course possible that the issue might be addressed in the upcoming Git for Windows version which uses POSIX semantics when renaming, which might fix the underlying issue.

That link redirected me to #391 which I think is incorrect? Could you double check the link? I think that may help me with my issue.

[darkfeline]

(for reference, I found the POSIX semantics discussion here: https://lore.kernel.org/all/[email protected]/T/)