Opening to your /Users/davis/ folder will near infinitely "discover interpreters" in the Console

[DavisVaughan]

@mine-cetinkaya-rundel did this out of curiosity, but it would be nice to look into what is going on here.

It look roughly 5 minutes to finally finish discovering interpreters - is it something we do in that folder, or is it unrelated that it takes this long?

[DavisVaughan]

It also takes 10-15 seconds to discover interpreters after she hits Close folder, when it typically only takes 1-2 seconds when she opens a "normal" file

[jmcphers]

I don't know for sure, but my guess would be that the Python extension is recursively searching the workspace folder for things like venvs.

[seeM]

Strange, I can only repro this in a release build but not in local dev.

Looking at the Python logs in Positron Version: 2024.02.0 (Universal) build 1514:

2024-02-12 10:13:49.445 [info] activatePositron: done!
2024-02-12 10:14:20.124 [info] pythonRuntimeDiscoverer: Starting Python runtime discoverer
<snip>
2024-02-12 10:17:02.616 [info] pythonRuntimeDiscoverer: skipping unsupported interpreter /Users/seem/.pyenv/versions/3.7.16/bin/python

The duration between the first two lines is from when the Python extension registers the runtime discoverer until Positron requests the first item. The last line is when Python's discovery is completed. I don't think it's the Python extension but could be wrong. (EDIT: Not sure why I said that! It's taking 2m42s according to those logs)

[seeM]

@wesm hit this too, Slack thread here and relevant info quoted below. We may want to consider moving this to Public Beta.

Here you can see the log of it spending 8 minutes painfully discovering interpreters:

https://gist.github.com/wesm/afa170618d6ba8182f663b046adf7586

I've been having some other slowness problems on this computer but unsure if it's related. I'll try blowing away as much application data as I can find and rebuilding the project from a fresh git clone tomorrow

This command takes over 20 seconds to run on my machine, but I can't tell if it has something to do with macOS's "system integrity protection" or the company's Crowdstrike software scanning Python binaries downloaded from the internet

conda run -n base --no-capture-output python ~/code/positron/extensions/positron-python/pythonFiles/get_output_via_markers.py -c "import ipykernel; print(ipykernel.__version__)"

The logs show that both conda and /usr/bin/python3 are slow so not specific to the environment manager.

[wesm]

We may want to consider moving this to Public Beta.

Yeah, this is a REALLY bad way for the application to fail. If I were an end user, an issue like this would probably be enough for me to bounce.

Some users may have lots of Python environments, so:

• The last environment used for a project should load right away without waiting on discovery to complete
• Slow environment discovery should not render the application so unusable

[jmcphers]

Wes, I have a PR in the works which should address both of those points. (It doesn't do anything about Python discovery being slow, though.)

[wesm]

@jmcphers that's great to know. I question whether it is necessary to re-check all Python environments every time the application starts up. It seems like it would be better to:

• Check an environment only ever once and record whether it has ipykernel in the application settings
• If in the future, launching an environment fails, it is nixed from the settings, and rechecked asynchronously in the future without blocking the user from getting up and running

This would convert environment discovery in the well-behaved case from an every-time cost to a one-time cost. I haven't looked yet at other open issues about this. Another option would be to only do the ipykernel check when trying to launch an environment, rather than trying to filter them all out up front.

I reinstalled macOS on my computer this weekend and things appear to be speedy again, but this seems to align with my anecdotal observation that macOS major releases are degrading the non-consumer user experience and making things worse for developers year over year.