Missing coverage in try-with-resources finally block

[Wouter684]

Describe the bug

When running a pitest on a class with a try-with-resources and a finally block, the logic in the finally block is not covered by the VoidMethodCallMutator. There is a test case which fails when the method call is removed, but when pitest removes that method call the mutation survives according to pitest.

Expected behavior

The mutation should be killed when the method call is removed and the pitest should have full coverage on that line, because the method call is covered in a test case.

Actual behavior

The mutation survives even though there is coverage to ensure that the removed line is executed.

How to Reproduce?

https://github.com/Wouter684/pitest-try-with-resources-bug

Run the pitest in the project. (mvn clean test org.pitest:pitest-maven:mutationCoverage)

Output of java -version

openjdk version "21.0.3" 2024-04-16 LTS
OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode)  

Quarkus version or git rev

3.17.5

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.9 (36645f6c9b5079805ea5009217e36f2cffd34256)
Maven home: /Users/<redacted>/.sdkman/candidates/maven/current
Java version: 21.0.3, vendor: Eclipse Adoptium, runtime: /Users/<redacted>/.sdkman/candidates/java/21.0.3-tem
Default locale: nl_NL, platform encoding: UTF-8
OS name: "mac os x", version: "14.6.1", arch: "aarch64", family: "mac"

Additional information

When running pitest with verbose logging, the result shows that for VoidMethodCallMutator, 3 mutations are generated but only 2 are killed. However, when looking through the verbose logs, only 2 of these mutations are actually executed and logged, the third never even occurs in the logging.

The entire mutation case that is not killed never shows up in the logging, as if the mutation has not been executed at all. Even though it shows up in the results as "No coverage" and the pitest fails over the missing coverage

[hcoles]

Thanks for the report. The underlying cause of this is the way that javac implements try blocks, creating duplicate inlined copies of the code. Pitest does contain some handling for this, but it hasn't been updated for many years and was always imperfect.

I'll take another look and see if it can be improved and if there have been any changes to the pattern of bytecode that recent versions of javac emit.

[hcoles]

@Wouter684 There seems to be an issue with the linked repo

pitest-try-with-resources-bug/src/test/java/org/example/MyClassTest.java:[47,85] cannot find symbol
[ERROR]   symbol:   method getFirst()

[Wouter684]

@Wouter684 There seems to be an issue with the linked repo

pitest-try-with-resources-bug/src/test/java/org/example/MyClassTest.java:[47,85] cannot find symbol
[ERROR]   symbol:   method getFirst()

That is strange, for me the test worked. Perhaps you are not yet using java 21 on your machine? I updated the repo to replace List.getFirst() with List.get(0) to ensure it works with older Java versions as well. Could you check if this fixed the issue for you?

[hcoles]

@Wouter684 ah, you're correct, it was picking up java 11. All fine with 21.