fix(region cache): x-amz-bucket-region can be missing for s3 compatible

[shcheklein]

In some cases x-amz-bucket-region. E.g. Yandex S3 returns these headers:

'server' = 'nginx'
'date' = 'Sat, 11 Jan 2025 18:24:12 GMT'
'content-type' = 'application/xml'
'connection' = 'keep-alive'
'keep-alive' = 'timeout=60'
'x-amz-request-id' = '427d58f44ef55822'

We do have a logic to disable region caching if custom endpoint of region are specified:

       for parameters in (config_kwargs, self.kwargs, init_kwargs, client_kwargs):
            for option in ("region_name", "endpoint_url"):
                if parameters.get(option):
                    self.cache_regions = False
                    break
        else:
            cache_regions = self.cache_regions

But it won't work if endpoint is specified via env var, or S3 config file (or some other complicated or even dynamic logic that boto is relying internally?). If someone has an idea how to get an efficient value for the endpoint URL - let me know.

[shcheklein]

@martindurant it would be amazing if you could review this please :) People are hitting this from the DVC usage also - https://discuss.dvc.org/t/dvc-external-output-dependencies-keyerror-x-amz-bucket-region/2432

[martindurant]

Sorry, I missed it. Looks perfect.

[shcheklein]

@martindurant no worries, thanks!