bz2 suffixes being used despite registry not having bz2 files

[benmoss]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

Using the --update flag seems to result in all of my packages that have .conda suffixes being replaced with .tar.bz2 suffixes. It seems to because of this line.

I took a look at fixing it but it seems like the overall strategy being used here of running conda list --json won't work since it doesn't give us the remote URL and so we're forced to re-assemble it from the channel and guess at what the extension is.

[maresb]

Ooh, that's quite nasty. Thanks for tracking down the cause! Unfortunately I don't have any particular ideas about how to fix this, but I'm open to suggestions.

[ctcjab]

Just had a minor outage and tracked it down to this bug (along with insufficient testing on our part).

Leaving the --update functionality in conda-lock in its current form is a nasty footgun. Is there any workaround? If not, maybe disable the --update flag until this is fixed? And if there is a workaround, at least modify the --update functionality to emit a loud warning like "This can result in broken lockfiles, see <link> for workaround. Continue?"

Maybe the workaround requires keeping the lockfile in version control, and doing something like

1. git diff conda-lock.yml
2. for each added .bz2 url parsed out of the diff
3. check if there is an associated removed .conda url in the diff
4. if so, rewrite the url back to a .conda suffix
5. possibly even check that the .conda URL does not 404 (and possibly that the .bz2 URL would have)

[maresb]

I'm really sorry about the trouble @ctcjab.

Is this something you'd be willing to contribute a workaround or fix for?

[ctcjab]

I'm not sure. Open to it, but some complicating factors I can think of:

• How practical would it be / how much time would you estimate it would take for a new contributor to this codebase (who is also not a conda expert, but is fluent in Python) to contribute a workaround or fix
• How long would you estimate it would be before the workaround or fix would be released
• Would it only be released in an incompatible new version, so would have associated migration costs to pick up

My manager is generally supportive of my making strategic contributions to the OSS we depend on. Just have to make a sound business case for it first.

[maresb]

Honestly, this codebase is despite my best efforts still fairly legacy. Also I don't know if there's an elegant fix. Unfortunately for some reason it seems like conda/micromamba disregard the extensions in the repodata cache. (To see what I mean, see mamba-org/mamba#3480 and the partial output of micromamba list --json:)

    {
        "base_url": "https://conda.anaconda.org/conda-forge",
        "build_number": 0,
        "build_string": "ha6fb4c9_0",
        "channel": "conda-forge",
        "dist_name": "zstd-1.5.6-ha6fb4c9_0",
        "name": "zstd",
        "platform": "linux-64",
        "version": "1.5.6"
    }

The status of the next major release is summarized in #725. Once that's published, I'm happy to cut semver-patch releases for individual PRs like the one we're talking about. I hope that helps. I wish it were easier.