Ruby LSP attempts to index ~/Library when being run with BBEdit

[etherbob]

Description

Reproduction steps

1. Start the Ruby LSP using BBEdit 15.1.x on MacOS Sonoma (14.x) with ruby language configured to use ~/.asdf/shims/ruby-lsp with and without the default stdio argument.
2. Tail the bbedit lsp log (/Users//Library/Containers/com.barebones.bbedit/Data/Library/Logs/BBEdit/LanguageServerProtocol-Ruby.txt)
3. Open a project window from the root of project (in this case a rails app). (bbedit . works well for this from a terminal window)
4. LSP fails to index see log at bottom of this summary (I substituted some project/username dirs with <username>, <sub-folder> etc to be conscientious about it being a work project.)

My apologies if non-vs-code use isn't fully supported yet. The BBEdit devs have been fairly responsive to my requests for help troubleshooting on their end and made some changes to try to keep the ruby-lsp server focused on the current workspace, but are really unsure as to why this is happening. I started a conversation in the Ruby DX slack with some good suggested troubleshooting ideas, but it got lost to history before I had time to revisit this issue.

================================================================================
2024-07-24 21:06:49.356: Application startup: 15.1.2 (15B37)
2024-07-24 21:06:49.356: macOS version 14.5 (23F79)
2024-07-24 21:06:49.356: Using guessed workspace context: file:///Users/<username>/dev/<sub-folder>/<project-folder>/
2024-07-24 21:06:49.356: Initialization parameters sent to server: {
    capabilities =     {
        textDocument =         {
            codeAction =             {
                codeActionLiteralSupport =                 {
                    codeActionKind =                     {
                        valueSet =                         (
                            info,
                            quickfix,
                            refactor,
                            source
                        );
                    };
                };
            };
            completion =             {
                completionItem =                 {
                    deprecatedSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    insertReplaceSupport = 1;
                    insertTextModeSupport =                     {
                        valueSet =                         (
                            1,
                            2
                        );
                    };
                    preselectSupport = 1;
                    snippetSupport = 1;
                };
            };
            documentSymbol =             {
                hierarchicalDocumentSymbolSupport = 1;
                labelSupport = 1;
            };
            hover =             {
                contentFormat =                 (
                    markdown,
                    plaintext
                );
            };
            onTypeFormatting =             {
            };
            publishDiagnostics =             {
                categorySupport = 1;
                codeActionsInline = 1;
                codeDescription = 1;
                dataSupport = 1;
                relatedInformation = 1;
            };
            rename =             {
                prepareSupport = 1;
                prepareSupportDefaultBehavior = 1;
            };
            signatureHelp =             {
                signatureInformation =                 {
                    activeParameterSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    parameterInformation =                     {
                        labelOffsetSupport = 1;
                    };
                };
            };
            synchronization =             {
                didSave = 1;
                willSave = 1;
            };
        };
        workspace =         {
            applyEdit = 1;
            configuration = 1;
            workspaceEdit =             {
                documentChanges = 0;
                failureHandling = abort;
            };
            workspaceFolders = 1;
        };
    };
    clientInfo =     {
        name = BBEdit;
        version = "15.1.2";
    };
    initializationOptions =     {
        diagnostics = 1;
    };
    rootUri = "file:///Users/<username>/dev/<sub-folder>/<project-folder>/";
    trace = verbose;
    workspaceFolders =     (
                {
            name = "<project-folder>";
            uri = "file:///Users/<username>/dev/<sub-folder>/<project-folder>/";
        }
    );
}
2024-07-24 21:06:49.356: Starting server, command: ruby-lsp, arguments: (
)
2024-07-24 21:06:49.356: Waiting for server startup to complete...
2024-07-24 21:06:49.687: stderr output from server: Ruby LSP> Skipping lockfile copies because there's no top level bundle

2024-07-24 21:06:49.703: stderr output from server: Ruby LSP> Running bundle install for the custom bundle. This may take a while...
Ruby LSP> Command: (bundle check || bundle install) 1>&2

2024-07-24 21:06:49.818: stderr output from server: The Gemfile's dependencies are satisfied

2024-07-24 21:06:50.133: stderr output from server: Initializing Ruby LSP v0.17.9...

2024-07-24 21:06:50.147: server startup response: {
    capabilities =     {
        codeActionProvider =         {
            resolveProvider = 1;
        };
        codeLensProvider =         {
        };
        completionProvider =         {
            completionItem =             {
                labelDetailsSupport = 1;
            };
            resolveProvider = 1;
            triggerCharacters =             (
                "/",
                "\"",
                "'",
                ":",
                "@",
                "."
            );
        };
        definitionProvider = 1;
        diagnosticProvider =         {
            interFileDependencies = 0;
            workspaceDiagnostics = 0;
        };
        documentHighlightProvider = 1;
        documentLinkProvider =         {
        };
        documentOnTypeFormattingProvider =         {
            firstTriggerCharacter = "{";
            moreTriggerCharacter =             (
                "\n",
                "|",
                d
            );
        };
        documentSymbolProvider =         {
            hierarchicalDocumentSymbolSupport = 1;
            symbolKind =             {
                "value_set" =                 (
                    1,
                    2,
                    3,
                    4,
                    5,
                    6,
                    7,
                    8,
                    9,
                    10,
                    11,
                    12,
                    13,
                    14,
                    15,
                    16,
                    17,
                    18,
                    19,
                    20,
                    21,
                    22,
                    23,
                    24,
                    25,
                    26
                );
            };
        };
        experimental =         {
            "addon_detection" = 1;
        };
        foldingRangeProvider =         {
            lineFoldingOnly = 1;
        };
        hoverProvider =         {
        };
        inlayHintProvider =         {
        };
        positionEncoding = "utf-16";
        selectionRangeProvider = 1;
        semanticTokensProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                }
            );
            full =             {
                delta = 0;
            };
            legend =             {
                tokenModifiers =                 (
                    declaration,
                    definition,
                    readonly,
                    static,
                    deprecated,
                    abstract,
                    async,
                    modification,
                    documentation,
                    "default_library"
                );
                tokenTypes =                 (
                    namespace,
                    type,
                    class,
                    enum,
                    interface,
                    struct,
                    typeParameter,
                    parameter,
                    variable,
                    property,
                    enumMember,
                    event,
                    function,
                    method,
                    macro,
                    keyword,
                    modifier,
                    comment,
                    string,
                    number,
                    regexp,
                    operator,
                    decorator
                );
            };
            range = 1;
        };
        signatureHelpProvider =         {
            triggerCharacters =             (
                "(",
                " ",
                ","
            );
        };
        textDocumentSync =         {
            change = 2;
            openClose = 1;
        };
        typeHierarchyProvider =         {
        };
        workspaceSymbolProvider = 1;
    };
    formatter = none;
    serverInfo =     {
        name = "Ruby LSP";
        version = "0.17.9";
    };
}
2024-07-24 21:06:50.147: ...completed server startup.
2024-07-24 21:06:50.148: stderr output from server: Finished initializing Ruby LSP!

2024-07-24 21:06:50.722: Server message (error): Error while indexing: Operation not permitted - /Users/<username>/Library/Application Support/com.apple.LaunchServicesTemplateApp.dv

[andyw8]

Hi @etherbob

If you run ruby-lsp --doctor in the root of the project, do you see any errors?

[andyw8]

Also, I wonder if this relates to https://www.barebones.com/support/bbedit/AppSandboxing.html

[vinistock]

The only paths we try to index automatically are gem installation paths and the files inside the workspace.

If you run this

gem env path

Do any of the entries include anything under ~/Library? Do you happen to have an .index.yml or other Ruby LSP indexing configuration? Any possible symlinks pointing to ~/Library?

We rely on Bundler for figuring out all of the paths we're going to index, so it might be worth checking Bundler configuration too.

[etherbob]

@andyw8

I can run ruby-lsp --doctor and it only indexes my local gems and my project files...
Same with running the command inside a BBEdit Shell Worksheet

I wonder if there's something in the config BBEdit is passing in:

2024-07-24 21:06:49.356: Initialization parameters sent to server: {
    capabilities =     {
        textDocument =         {
            codeAction =             {
                codeActionLiteralSupport =                 {
                    codeActionKind =                     {
                        valueSet =                         (
                            info,
                            quickfix,
                            refactor,
                            source
                        );
                    };
                };
            };
            completion =             {
                completionItem =                 {
                    deprecatedSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    insertReplaceSupport = 1;
                    insertTextModeSupport =                     {
                        valueSet =                         (
                            1,
                            2
                        );
                    };
                    preselectSupport = 1;
                    snippetSupport = 1;
                };
            };
            documentSymbol =             {
                hierarchicalDocumentSymbolSupport = 1;
                labelSupport = 1;
            };
            hover =             {
                contentFormat =                 (
                    markdown,
                    plaintext
                );
            };
            onTypeFormatting =             {
            };
            publishDiagnostics =             {
                categorySupport = 1;
                codeActionsInline = 1;
                codeDescription = 1;
                dataSupport = 1;
                relatedInformation = 1;
            };
            rename =             {
                prepareSupport = 1;
                prepareSupportDefaultBehavior = 1;
            };
            signatureHelp =             {
                signatureInformation =                 {
                    activeParameterSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    parameterInformation =                     {
                        labelOffsetSupport = 1;
                    };
                };
            };
            synchronization =             {
                didSave = 1;
                willSave = 1;
            };
        };
        workspace =         {
            applyEdit = 1;
            configuration = 1;
            workspaceEdit =             {
                documentChanges = 0;
                failureHandling = abort;
            };
            workspaceFolders = 1;
        };
    };
    clientInfo =     {
        name = BBEdit;
        version = "15.1.2";
    };
    initializationOptions =     {
        diagnostics = 1;
    };
    rootUri = "file:///Users/<username>/dev/<sub-folder>/<project-folder>/";
    trace = verbose;
    workspaceFolders =     (
                {
            name = "<project-folder>";
            uri = "file:///Users/<username>/dev/<sub-folder>/<project-folder>/";
        }
    );
}

[etherbob]

@vinistock interestingly

gem env path
/Users/<username>/.gem/ruby/2.6.0:/Library/Ruby/Gems/2.6.0:/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/gems/2.6.0

that's from in a BBEdit shell worksheet which I think would be the same env as bbedit tries to shell out to ruby-lsp

[etherbob]

I wonder if BBEdit is running it with a current working directory of ~ and assuming the workspace setting will override it. Is there a flag I can pass to ruby-lsp to not auto-index the current working dir?

[vinistock]

Not really, these are the indexing configurations we expose.

My suspicion is that this is related to some intersection of BBEdit and ASDF. Something might be overriding the GEM_HOME or GEM_PATH and we end up trying to index Library.

[etherbob]

I was able to get solargraph (BBEdit's default configured ruby language server) working, but I don't know if it has the same GEM_HOME/GEM_PATH behavior, or what it does when it hits a permission error. I'm going to see if I can wrap the ruby-lsp binary in a shell script so I can echo those environment values. My zsh scripting skills are... minimal though, so we'll see how this goes.

[etherbob]

@vinistock you called it inside BBEdit's shell worksheet running gem env

gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.0.3.1
  - RUBY VERSION: 2.6.10 (2022-04-12 patchlevel 210) [universal.arm64e-darwin23]
  - INSTALLATION DIRECTORY: /Library/Ruby/Gems/2.6.0
  - USER INSTALLATION DIRECTORY: /Users/drew/.gem/ruby/2.6.0
  - RUBY EXECUTABLE: /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /usr/local/bin
  - SPEC CACHE DIRECTORY: /Users/drew/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /Library/Ruby/Site
  - RUBYGEMS PLATFORMS:
    - ruby
    - universal-darwin-23
  - GEM PATHS:
     - /Library/Ruby/Gems/2.6.0
     - /Users/drew/.gem/ruby/2.6.0
     - /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/lib/ruby/gems/2.6.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /usr/local/bin
     - /System/Cryptexes/App/usr/bin
     - /usr/bin
     - /bin
     - /usr/sbin
     - /sbin
     - /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin
     - /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin
     - /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
     - /Users/drew/.asdf/shims
     - /Users/drew/.asdf/bin
     - /usr/local/sbin
     - /opt/homebrew/bin
     - /opt/homebrew/sbin

I'll figure out how to override it in .zshenv to something sane. Thanks for helping me run this to ground.

[andyw8]

I'll close this for now since the problem seems to be not due to Ruby LSP, but feel free to update the issue with your findings.

[etherbob]

it's being set by /etc/profile

/usr/libexec/path_helper -s gets prepended to the path

which puts system ruby ahead of the one set by asdf in zshenv

BBEdit runs a non-interactive login shell so zshrc (which must come after /etc/profile) gets run and puts the asdf supplied ruby back in front.

[etherbob]

@andyw8 I did a little more experimenting and even with the right env it still tries to index my home dir when started up from a non-interactive login shell (what BBEdit uses to run the language servers)

I made a test script (lang-serv.sh)

#! /bin/zsh
echo $PATH >&2
which ruby >&2
ruby --version >&2
gem env >&2
pwd
ruby-lsp

pwd prints no output but the first time it gets an index command from BBEdit it attempts to index my home dir

same thing happens if I add --doctor to my script.

if I add a line that cd's to my main project directory everything works fine and dandy.

[etherbob]

Ok looking at https://github.com/Shopify/ruby-lsp/blob/main/lib/ruby_indexer/lib/ruby_indexer/configuration.rb#L27 it looks like ruby-lsp always attempts to index the Dir.pwd in addition to whatever workspace configs are passed in, but maybe I'm misreading this?

[ascarter]

@etherbob did you find a solution? I think your observation is correct. It also doesn't look like BBEdit provides the usual env vars when you normally run a script with the file path, etc.

[vinistock]

@ascarter what errors are you seeing with BBEdit? The one originally reported in this issue has been fixed already, so maybe it's something else.

[ascarter]

I see it scanning the working directory which unfortunately in BBEdit looks like the home directory not the project directory in question. I tried using a script to be the lsp call that changes the directory and that does seem to work (although BBEdit doesn't seem to show documentation for the matches). This is the error I get using rbenv to set my global ruby to 3.4.1 with ruby-lsp gem installed. It looks like it is scanning subdirectories of my home directory and for whatever reason it has permission error. Maybe the one thing that should change in ruby-lsp is to use the rootUri sent in the request?

================================================================================
2025-01-09 10:08:29.942: Application startup: 15.1.3 (15B62)
2025-01-09 10:08:29.942: macOS version 15.2 (24C101)
2025-01-09 10:08:29.942: Using guessed workspace context: file:///Users/acarter/Developer/ruby/hello/
2025-01-09 10:08:29.943: Initialization parameters sent to server: {
    capabilities =     {
        textDocument =         {
            codeAction =             {
                codeActionLiteralSupport =                 {
                    codeActionKind =                     {
                        valueSet =                         (
                            info,
                            quickfix,
                            refactor,
                            source
                        );
                    };
                };
            };
            completion =             {
                completionItem =                 {
                    deprecatedSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    insertReplaceSupport = 1;
                    insertTextModeSupport =                     {
                        valueSet =                         (
                            1,
                            2
                        );
                    };
                    preselectSupport = 1;
                    snippetSupport = 1;
                };
            };
            documentSymbol =             {
                hierarchicalDocumentSymbolSupport = 1;
                labelSupport = 1;
            };
            hover =             {
                contentFormat =                 (
                    markdown,
                    plaintext
                );
            };
            onTypeFormatting =             {
            };
            publishDiagnostics =             {
                categorySupport = 1;
                codeActionsInline = 1;
                codeDescription = 1;
                dataSupport = 1;
                relatedInformation = 1;
            };
            rename =             {
                prepareSupport = 1;
                prepareSupportDefaultBehavior = 1;
            };
            signatureHelp =             {
                signatureInformation =                 {
                    activeParameterSupport = 1;
                    documentationFormat =                     (
                        markdown,
                        plaintext
                    );
                    parameterInformation =                     {
                        labelOffsetSupport = 1;
                    };
                };
            };
            synchronization =             {
                didSave = 1;
                willSave = 1;
            };
        };
        workspace =         {
            applyEdit = 1;
            configuration = 1;
            workspaceEdit =             {
                documentChanges = 0;
                failureHandling = abort;
            };
            workspaceFolders = 1;
        };
    };
    clientInfo =     {
        name = BBEdit;
        version = "15.1.3";
    };
    initializationOptions =     {
        diagnostics = 1;
    };
    rootUri = "file:///Users/acarter/Developer/ruby/hello/";
    trace = verbose;
    workspaceFolders =     (
                {
            name = hello;
            uri = "file:///Users/acarter/Developer/ruby/hello/";
        },
                {
            name = dotfiles;
            uri = "file:///Users/acarter/.config/dotfiles/";
        },
                {
            name = BBEdit;
            uri = "file:///Users/acarter/Library/Containers/com.barebones.bbedit/Data/Library/Logs/BBEdit/";
        }
    );
}
2025-01-09 10:08:29.943: Starting server, command: ruby-lsp, arguments: (
)
2025-01-09 10:08:29.943: Waiting for server startup to complete...
2025-01-09 10:08:31.594: stderr output from server: Ruby LSP> Skipping lockfile copies because there's no top level bundle

2025-01-09 10:08:31.744: stderr output from server: Ruby LSP> Running bundle install for the composed bundle. This may take a while...
Ruby LSP> Command: ((/Users/acarter/.rbenv/versions/3.4.1/bin/ruby /Users/acarter/.rbenv/versions/3.4.1/bin/bundle check && /Users/acarter/.rbenv/versions/3.4.1/bin/ruby /Users/acarter/.rbenv/versions/3.4.1/bin/bundle update ruby-lsp debug) || /Users/acarter/.rbenv/versions/3.4.1/bin/ruby /Users/acarter/.rbenv/versions/3.4.1/bin/bundle install) 1>&2

2025-01-09 10:08:32.301: stderr output from server: The Gemfile's dependencies are satisfied

2025-01-09 10:08:34.174: stderr output from server: Fetching gem metadata from https://rubygems.org/
2025-01-09 10:08:34.178: stderr output from server: .
2025-01-09 10:08:34.226: stderr output from server: .
2025-01-09 10:08:34.235: stderr output from server: .
2025-01-09 10:08:34.256: stderr output from server: .
2025-01-09 10:08:34.263: stderr output from server: .
2025-01-09 10:08:34.267: stderr output from server: .
2025-01-09 10:08:34.273: stderr output from server: .
2025-01-09 10:08:34.276: stderr output from server: .
2025-01-09 10:08:34.344: stderr output from server: 

2025-01-09 10:08:34.363: stderr output from server: Resolving dependencies...

2025-01-09 10:08:34.378: stderr output from server: Resolving dependencies...

2025-01-09 10:08:34.617: stderr output from server: Fetching sorbet-runtime 0.5.11742 (was 0.5.11734)

2025-01-09 10:08:34.970: stderr output from server: Installing sorbet-runtime 0.5.11742 (was 0.5.11734)

2025-01-09 10:08:35.027: stderr output from server: Bundler attempted to update ruby-lsp but its version stayed the same
Bundler attempted to update debug but its version stayed the same
Bundle updated!

2025-01-09 10:08:39.786: Server message: Initializing Ruby LSP v0.23.2...
2025-01-09 10:08:39.787: server startup response: {
    "bundle_env" = "<null>";
    capabilities =     {
        codeActionProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                }
            );
            resolveProvider = 1;
        };
        codeLensProvider =         {
        };
        completionProvider =         {
            completionItem =             {
                labelDetailsSupport = 1;
            };
            resolveProvider = 1;
            triggerCharacters =             (
                "/",
                "\"",
                "'",
                ":",
                "@",
                ".",
                "=",
                "<",
                "$"
            );
        };
        definitionProvider = 1;
        diagnosticProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                }
            );
            interFileDependencies = 0;
            workspaceDiagnostics = 0;
        };
        documentHighlightProvider = 1;
        documentLinkProvider =         {
        };
        documentOnTypeFormattingProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                }
            );
            firstTriggerCharacter = "{";
            moreTriggerCharacter =             (
                "\n",
                "|",
                d
            );
        };
        documentRangeFormattingProvider = 1;
        documentSymbolProvider =         {
        };
        experimental =         {
            "addon_detection" = 1;
        };
        foldingRangeProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                }
            );
        };
        hoverProvider =         {
        };
        inlayHintProvider =         {
        };
        positionEncoding = "utf-16";
        referencesProvider = 1;
        renameProvider =         {
            prepareProvider = 1;
        };
        selectionRangeProvider = 1;
        semanticTokensProvider =         {
            documentSelector =             (
                                {
                    language = ruby;
                },
                                {
                    language = erb;
                }
            );
            full =             {
                delta = 1;
            };
            legend =             {
                tokenModifiers =                 (
                    declaration,
                    definition,
                    readonly,
                    static,
                    deprecated,
                    abstract,
                    async,
                    modification,
                    documentation,
                    "default_library"
                );
                tokenTypes =                 (
                    namespace,
                    type,
                    class,
                    enum,
                    interface,
                    struct,
                    typeParameter,
                    parameter,
                    variable,
                    property,
                    enumMember,
                    event,
                    function,
                    method,
                    macro,
                    keyword,
                    modifier,
                    comment,
                    string,
                    number,
                    regexp,
                    operator,
                    decorator
                );
            };
            range = 1;
        };
        signatureHelpProvider =         {
            triggerCharacters =             (
                "(",
                " ",
                ","
            );
        };
        textDocumentSync =         {
            change = 2;
            openClose = 1;
        };
        typeHierarchyProvider =         {
        };
        workspaceSymbolProvider = 1;
    };
    "degraded_mode" = 0;
    formatter = none;
    serverInfo =     {
        name = "Ruby LSP";
        version = "0.23.2";
    };
}
2025-01-09 10:08:39.787: ...completed server startup.
2025-01-09 10:08:39.787: Server message: Auto detected formatter: none
2025-01-09 10:08:39.787: Server message: Auto detected linters: 
2025-01-09 10:08:39.790: Server message: Detected test library: unknown
2025-01-09 10:08:39.791: Server message: Finished initializing Ruby LSP!
2025-01-09 10:08:42.875: Server message (error): Error while indexing (see [troubleshooting steps](https://shopify.github.io/ruby-lsp/troubleshooting#indexing)): Operation not permitted - /Users/acarter/Library/Accounts

[vinistock]

We indeed regressed and ended up re-introducing this bug. The changes in #3034 will fix it. @ascarter thank you for reporting!

[ascarter]

Great - thanks @vinistock!