Test explorer improvements

[JohnnyFasching]

I would have some points for the improvement of the test explorer

• Group tests und the project they were found in
  It would make a lot of sense to group the tests with the same project structure the solution has, this would enhance the readability of the tests

• Partial or automatic refresh of the test explorer
  The refresh of all the tests and build all the projects every time you change or add some tests for one project just makes no sense. If it is possible to check what changes were made and build the project accordingly this would improve performance dramatically or otherwise a partial refresh of the explorer would also help

[baronfel]

Great feedback! cc'ing @farlee2121 for ideas/thoughts on these.

[farlee2121]

• Tests should already be grouped by project. I just double checked and my projects do appear that way. Screen shots and/or a repo would be appreciated if you're seeing different behavior
• The explorer does attempt to update from changes in code analysis, though there are condition where it can't translate code analysis into explorer changes. It also detects changes in test results. If you run a grouping of tests, it should discover and display changes in that test grouping when it displays results.

[farlee2121]

@JohnnyFasching Are you able to describe more specifically how you are expecting the explorer to update, but find missing?

[JohnnyFasching]

Sorry you are right the Tests are already grouped by the Project. It could be because we are using TickSpec for our tests that the automatic refresh isn't working for our scenarios.

This could also lead to the way the tests are displayed in our explorer. They Are all in on section called Feature. I will try to set up a small sample project so I can show you exactly how it looks in such a project.

[farlee2121]

Ionide has its own code analysis that detects test locations. I'd guess TickSpec isn't currently supported.
I think Visual Studio has a more modular code location approach through the test adapters (based on a recent discussion for expecto code locations).

Maybe the answer here is to understand how the Visual Studio code location works and see if we can use the same adapters for Ionide

[JohnnyFasching]

I could check that and if we could use this approach to smoothen also the experience with TickSpec. This would increase the readability a lot, as the grouping works fine for each project, but the grouping of the test cases defined in the feature files are ignored

[MangelMaxime]

Another improvements that could be made is to add support for continuous.

It allows you to rerun one, several or all the tests when your source code changes. This is great to get a fast feedback loop.

[farlee2121]

A lot of pieces are already there for continuous. We can run tests by project, filtered to certain test groups, and update the explorer based on test run results.

We also detect changes in test code and map those changes to the test explorer.
But, I generally expect continuous mode to run tests whenever covered code is changed. I'm not sure if we have the tools for that kind of code analysis.

[jkone27]

here also some feedback on menu improvements, seems we are missing "go to text" in C# tests in mixed slns (go to source of tests code) vs C# dev kit. #2010

[MangelMaxime]

Another improvement I can see, is perhaps we could try to compact the tests tree.

For example, when using MSTest with the following code:

module Fable.Package.SDK.Tests

open Microsoft.VisualStudio.TestTools.UnitTesting
open SimpleExec
open System.IO

[<TestClass>]
type All () =

    [<TestMethod>]
    member this.TestMethodPassing() =
        Assert.IsTrue(true)

I get this tree:

Having to click 7 times in order to access the actual test is annoying.

What do you think about compacting the tree when a node only have a single children?

In the example above it would give us something like:

Fable.Package.SDK.Tests (net8.0)
	Fable.Package.SDK.Tests.All	<- I don't think we can remove this one
		TestMethodPassing

I am just starting to explore the trx format, depending on my finding in it this feature could be more or less easy to do. For example, I see className="Fable.Package.SDK.Tests+All" name="TestMethodPassing" which makes it easy to identify Fable.Package.SDK.Tests module and could avoid a custom logic for compacting in a naive way as a first approach.

But I wanted to check with others maintainers of Ionide, if there was an interest in compacting the tree to make it easier to navigate.

[baronfel]

I love this idea - you can already see it in the File Explorer's tree view. I think we should try to adopt this UX convention everywhere it makes sense, so at minimum here and in the Solution Explorer.

[farlee2121]

Agreed. That would be a very handy feature.