-
-
Notifications
You must be signed in to change notification settings - Fork 279
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test explorer stability: Use test results as main truth source for explorer #1874
Commits on Jul 8, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 425764d - Browse repository at this point
Copy the full SHA 425764dView commit details -
Configuration menu - View commit details
-
Copy full SHA for fcff548 - Browse repository at this point
Copy the full SHA fcff548View commit details -
Configuration menu - View commit details
-
Copy full SHA for 607d328 - Browse repository at this point
Copy the full SHA 607d328View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2adb235 - Browse repository at this point
Copy the full SHA 2adb235View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0077521 - Browse repository at this point
Copy the full SHA 0077521View commit details -
Configuration menu - View commit details
-
Copy full SHA for d43615c - Browse repository at this point
Copy the full SHA d43615cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 3614b5d - Browse repository at this point
Copy the full SHA 3614b5dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 8fcc22d - Browse repository at this point
Copy the full SHA 8fcc22dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1926480 - Browse repository at this point
Copy the full SHA 1926480View commit details -
Update test tree based on test results
This is especially helpful for tests that don't have code locations. The user can run a subset of tests and changes to that portion of the test tree will be reflected
Configuration menu - View commit details
-
Copy full SHA for fb8eea1 - Browse repository at this point
Copy the full SHA fb8eea1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5330dc4 - Browse repository at this point
Copy the full SHA 5330dc4View commit details -
Refactored state updates around individual projects. This reduces mapping since tests execution is actualized by project
Configuration menu - View commit details
-
Copy full SHA for 3fac404 - Browse repository at this point
Copy the full SHA 3fac404View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4b8587b - Browse repository at this point
Copy the full SHA 4b8587bView commit details -
Generalize test hierarchy builder
This is a preperatory refactor in anticipation of changing test discovery from trx to --list-tests
Configuration menu - View commit details
-
Copy full SHA for d0f2627 - Browse repository at this point
Copy the full SHA d0f2627View commit details -
Support nested class hierarchies
I.e. when there are mixed + and . separators in the test name
Configuration menu - View commit details
-
Copy full SHA for f0a0f98 - Browse repository at this point
Copy the full SHA f0a0f98View commit details -
Configuration menu - View commit details
-
Copy full SHA for c3996ad - Browse repository at this point
Copy the full SHA c3996adView commit details -
Change filters to balance speed and discovery
The previous filter found all known test items to run from the explorer and matched them exactly. But this won't allow the explorer to discover tests from test results. If the user selects a group of tests, then it should run all tests in that group even if they aren't in the explorer then updated the explorer if there are differences. This is especially critical for C# tests which can't match code locations and update that way.
Configuration menu - View commit details
-
Copy full SHA for 13cde47 - Browse repository at this point
Copy the full SHA 13cde47View commit details -
Write test console output to Test Result log.
Improves discoverability of error and failure messages, especially for tests without code locations.
Configuration menu - View commit details
-
Copy full SHA for 3c1cf86 - Browse repository at this point
Copy the full SHA 3c1cf86View commit details -
Stablize initial test explorer discovery
Building projects in parallel was leading to build failures and tests being discovered incorrectly. Also effected explorer refresh.
Configuration menu - View commit details
-
Copy full SHA for 4261a47 - Browse repository at this point
Copy the full SHA 4261a47View commit details -
Fix resource starvation for solutions with many projects
Restoring in parallel was spawning too many promises (i.e. 87 in FAKE solution) and causing computer lag
Configuration menu - View commit details
-
Copy full SHA for 72428a2 - Browse repository at this point
Copy the full SHA 72428a2View commit details -
Add constrained promise parallelism
Added a new method that runs at most a certain number of Promises at a time. Allows a balance of resource usage safety and speed. It's faster than sequential, but know we won't end up queuing so many promises we end up with resource starvation. I.e. restoring against 87 projects in the FAKE repo
Configuration menu - View commit details
-
Copy full SHA for d4e8ba7 - Browse repository at this point
Copy the full SHA d4e8ba7View commit details -
Fix small collection errors in executeWithMaxParallel
take and skip are unsafe operations, causing executeWithMaxParallel to error if the input collection is smaller than the max parallel count
Configuration menu - View commit details
-
Copy full SHA for b7d0d25 - Browse repository at this point
Copy the full SHA b7d0d25View commit details -
Configuration menu - View commit details
-
Copy full SHA for e55775c - Browse repository at this point
Copy the full SHA e55775cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 765b710 - Browse repository at this point
Copy the full SHA 765b710View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5714bf4 - Browse repository at this point
Copy the full SHA 5714bf4View commit details -
Configuration menu - View commit details
-
Copy full SHA for e4b0026 - Browse repository at this point
Copy the full SHA e4b0026View commit details -
Configuration menu - View commit details
-
Copy full SHA for 01a70cc - Browse repository at this point
Copy the full SHA 01a70ccView commit details -
Run tests and test discovery against a single target framework
This fixes indeterminism if a project has multiple target frameworks. Especially if it runs some tests only on certain targets. Always using the first target framework aligns with other Ionide behavior.
Configuration menu - View commit details
-
Copy full SHA for 80580fb - Browse repository at this point
Copy the full SHA 80580fbView commit details -
The top level of the test explorer is organized by project. This will simplify some efforts like progressive loading and supporting multiple Target Frameworks
Configuration menu - View commit details
-
Copy full SHA for 8e9e4f0 - Browse repository at this point
Copy the full SHA 8e9e4f0View commit details -
Add target framework to project test items
Paves the way for multiple Target frameworks and provides context so users can figure out why the test count is different than the console or other test explorers See this discussion: ionide#1874 (comment)
Configuration menu - View commit details
-
Copy full SHA for 64d0ab7 - Browse repository at this point
Copy the full SHA 64d0ab7View commit details -
Preserve trailing space in test names using list discovery
Trailing spaces were causing test run results to not match up with the expected test items. The same issue still exists for leading spaces in a test name. However, It's unclear how consistent the indentation of test names would be in the console output. Rather than increase the risk of failure, I opted to retain the potential error for leading spaces. Leading spaces in a test name should not be common use case and is most likely an mistype
Configuration menu - View commit details
-
Copy full SHA for 7b45b7b - Browse repository at this point
Copy the full SHA 7b45b7bView commit details -
Ask user to restore before test discovery
Moving responsibility for initial restore to the user. It was proving difficult to consistently report restore errors, and, since we don't have access to optimal build orders, restore was adding a lot of time to test discovery when we nominally expect it to already be done.
Configuration menu - View commit details
-
Copy full SHA for a411c12 - Browse repository at this point
Copy the full SHA a411c12View commit details
Commits on Jul 9, 2023
-
Improve error reporting during test discovery
Detect build failures and when a test run likely failed to complete
Configuration menu - View commit details
-
Copy full SHA for d279a97 - Browse repository at this point
Copy the full SHA d279a97View commit details
Commits on Aug 3, 2023
-
Limit parallel test project execution
Prevents performance issues experienced with a large number of test projects.
Configuration menu - View commit details
-
Copy full SHA for 95a2b78 - Browse repository at this point
Copy the full SHA 95a2b78View commit details -
Linear build when running tests & centralize enqueue
Parallel project builds can conflict if they have mutual dependencies. This was more prominent during discovery and already fixed there, but it can also happen when running tests. Linearizing the build and limiting parallelism also changed the test status UX. Explicitly showing all enqueued right away makes it more intuitive that that the explorer responded correctly to the user's run request
Configuration menu - View commit details
-
Copy full SHA for 6bbc1a4 - Browse repository at this point
Copy the full SHA 6bbc1a4View commit details
Commits on Aug 8, 2023
-
Respect cancel tokens in explorer
Pass cancel tokens through the test, build, and refresh call chains. Attempt to kill any long-running processes spawned (i.e. dotnet test, msbuild) Unfortunately, secondary processes (like testhost) aren't cancelling when their parent does.
Configuration menu - View commit details
-
Copy full SHA for 6567b12 - Browse repository at this point
Copy the full SHA 6567b12View commit details -
Clarify implicit test framework property
Used to for workarounds specific to NUnit.
Configuration menu - View commit details
-
Copy full SHA for 88c642d - Browse repository at this point
Copy the full SHA 88c642dView commit details -
Fix NUnit filters with spaces when code loads before discovery
NUnit can't handle spaces in test filter expressions. The mitigation for it depends on knowing the test is an NUnit test. However, that information was only available if code-sourced test info is merged after code discovery. This makes that info available from trx discovery too.
Configuration menu - View commit details
-
Copy full SHA for e882c87 - Browse repository at this point
Copy the full SHA e882c87View commit details
Commits on Aug 19, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 61e938c - Browse repository at this point
Copy the full SHA 61e938cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 538f8c6 - Browse repository at this point
Copy the full SHA 538f8c6View commit details