[performance] Sometimes Eclipse goes into a state where it allocates memory like crazy and only a restart helps

[jubax]

I created a corresponding issue for JDT, but they think it is a lsp4e problem.

I am just linking to the corresponding JDT issue: eclipse-jdt/eclipse.jdt.core#3634. That issue shows screenshots and attaches a stack trace.

The last comment from a JDT contributor says that the stack trace points to CodeMining and the CodeMining stacktrace given points to lsp4e.

"ForkJoinPool.commonPool-worker-35" #633 [76891] daemon prio=6 os_prio=31 cpu=159560.22ms elapsed=443.99s tid=0x000000011736fe00 nid=76891 waiting for monitor entry  [0x00000003420b6000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at org.eclipse.debug.internal.core.LaunchManager.getLaunches(LaunchManager.java:1385)
        - locked <0x000000070424c3e8> (a java.util.ArrayList)
        at org.eclipse.lsp4e.debug.breakpoints.TextSelectionToIVariable.hasDAPTarget(TextSelectionToIVariable.java:112)
        at org.eclipse.lsp4e.debug.breakpoints.TextSelectionToIVariable.getVariableFor(TextSelectionToIVariable.java:53)
        at org.eclipse.lsp4e.debug.breakpoints.TextSelectionToIVariable.getAdapter(TextSelectionToIVariable.java:42)
        at org.eclipse.core.internal.runtime.AdapterManager.lambda$13(AdapterManager.java:310)
        at org.eclipse.core.internal.runtime.AdapterManager$$Lambda/0x000000700133b2c8.apply(Unknown Source)
        at java.util.stream.ReferencePipeline$3$1.accept([email protected]/ReferencePipeline.java:197)
        at java.util.stream.ReferencePipeline$2$1.accept([email protected]/ReferencePipeline.java:179)
        at java.util.stream.ReferencePipeline$3$1.accept([email protected]/ReferencePipeline.java:197)
        at java.util.ArrayList$ArrayListSpliterator.tryAdvance([email protected]/ArrayList.java:1685)
        at java.util.stream.ReferencePipeline.forEachWithCancel([email protected]/ReferencePipeline.java:129)
        at java.util.stream.AbstractPipeline.copyIntoWithCancel([email protected]/AbstractPipeline.java:527)
        at java.util.stream.AbstractPipeline.copyInto([email protected]/AbstractPipeline.java:513)
        at java.util.stream.AbstractPipeline.wrapAndCopyInto([email protected]/AbstractPipeline.java:499)
        at java.util.stream.FindOps$FindOp.evaluateSequential([email protected]/FindOps.java:150)
        at java.util.stream.AbstractPipeline.evaluate([email protected]/AbstractPipeline.java:234)
        at java.util.stream.ReferencePipeline.findFirst([email protected]/ReferencePipeline.java:647)
        at org.eclipse.core.internal.runtime.AdapterManager.getAdapter(AdapterManager.java:312)
        at org.eclipse.core.internal.runtime.AdapterManager.loadAdapter(AdapterManager.java:336)
        at org.eclipse.core.runtime.Adapters.queryAdapterManager(Adapters.java:150)
        at org.eclipse.core.runtime.Adapters.adapt(Adapters.java:86)
        at org.eclipse.core.runtime.Adapters.adapt(Adapters.java:115)
        at org.eclipse.debug.internal.ui.codemining.DebugValueCodeMiningProvider.lambda$1(DebugValueCodeMiningProvider.java:69)
        at org.eclipse.debug.internal.ui.codemining.DebugValueCodeMiningProvider$$Lambda/0x0000007002550408.get(Unknown Source)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run([email protected]/CompletableFuture.java:1768)
        at java.util.concurrent.CompletableFuture$AsyncSupply.exec([email protected]/CompletableFuture.java:1760)
        at java.util.concurrent.ForkJoinTask.doExec([email protected]/ForkJoinTask.java:387)
        at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec([email protected]/ForkJoinPool.java:1312)
        at java.util.concurrent.ForkJoinPool.scan([email protected]/ForkJoinPool.java:1843)
        at java.util.concurrent.ForkJoinPool.runWorker([email protected]/ForkJoinPool.java:1808)
        at java.util.concurrent.ForkJoinWorkerThread.run([email protected]/ForkJoinWorkerThread.java:188)

Thank you.

[jonahgraham]

On a quick exam to me it looks like each individual part (TextSelectionToIVariable) and (DebugValueCodeMiningProvider) are fairly reasonable. But the combination is pretty bad.

This line: org.eclipse.debug.internal.ui.codemining.DebugValueCodeMiningProvider.lambda$1(DebugValueCodeMiningProvider.java:69) is doing an adapt that is fairly intensive if you have multiple active launches. And since it is run over every line of the file (visible only?) the multiplier is pretty big. That may be leading to why there are 2.6 million live CompleteableFutures (eclipse-jdt/eclipse.jdt.core#3634 (comment)) which certainly an indication of a problem. I can't imagine a reasonable reason to have so many.

@jubax do you happen to have particularly large or many files open? Do you have many launches in the Debug view, even terminated ones?

Is it possible to trace the creation of CompletableFutures to confirm why they are being created in such quantity?

[mickaelistria]

It is known that the current approach to showing debug values with DAP is currently way too expensive; that's why it's marked as EXPERIMENTAL in preferences.
What we're missing is a way to determine from a file the interesting values to show and where. The current implementation (reading the whole file and passing bits to TextSelectionToIVariable) is way too greedy.

[jubax]

[...]
@jubax do you happen to have particularly large or many files open? Do you have many launches in the Debug view, even terminated ones?

I guess I typically have about 20 Java files open. The files are long and complicated, but probably not exceptionally long. I have probably about 300 Debug configurations (which I never use because I usually start debug sessions from a specific test). Is that what you meant by "launches in the Debug view"? I can delete them all and see if I still run into that problem.

[jubax]

It is known that the current approach to showing debug values with DAP is currently way too expensive; that's why it's marked as EXPERIMENTAL in preferences. What we're missing is a way to determine from a file the interesting values to show and where. The current implementation (reading the whole file and passing bits to TextSelectionToIVariable) is way too greedy.

You mean the option "Run/Debug -> Show debug values inline on text editors (Experimental)"? This is currently active on my system, but I have forgotten about it as it does not work for Java.

I will deselect that option. Maybe this already helps ... thank you.

[jonahgraham]

Is that what you meant by "launches in the Debug view"?

I was referring to the view shown in this screenshot. You do not need to delete the debug configurations you have, they don't have an impact on this issue.

It is known that the current approach to showing debug values with DAP is currently way too expensive;

I will deselect that option. Maybe this already helps ... thank you.

Thanks @mickaelistria for pointing us in the correct direction and highlighting the experimental nature of this.

I don't think there is much value in leaving this copy of the issue open, to me it looks like most of the work has to happen in DebugValueCodeMiningProvider anyway. Is there an open issue that can be linked to this one as @jubax's analysis here will certainly be useful for anyone tackling improving DebugValueCodeMiningProvider

[jubax]

Is that what you meant by "launches in the Debug view"?

I was referring to the view shown in this screenshot. You do not need to delete the debug configurations you have, they don't have an impact on this issue.
[...]

Thank you. I can confirm that I don't have many launches in the debug view.

[angelozerr]

I am experimenting to implement debug inline values in IntelliJ and I would like to share my studies.

First I have studied how vscode manage their debug inline values.

It uses 2 strategies:

• At first it tries to find if it exists an inline value provider. This provider is theoricly implemented with a language server lsp inlineValues. But to be honnest with you I have never seen a language server which implements this feature. An inline provider is an API from vscode and some vscode extension like vscode debug java registers a vscode inline provider which consumes a custom dap request.

• if the first strategy fails, vscode uses a second strategy: it uses the document model of the editor which is generally managed by textmate. It is this strategy that I am trying to implement and I think tm4e should provide also this strategy to retrieve proper textmate token which matches a variable name to retrieve the proper offset. In debugging context, the TMModel should be already ready since highlighting should be done before the debugging start. IMHO it should be faster than trying retrieving variable char by char and more precise.

I wanted just to share those ideas. Hope it will help you.

[mickaelistria]

Thanks for sharing @angelozerr . Indeed, a strategy that would first retrieve from the debug context the list of variable names that are relevant in this context and then showing only the values for those variable names would allow to skip a lot of string processing. We'd basically go from O(tokensInFile) evaluations to O(variableCount) evaluations, which is ridiculously less.