[Bug]: Dead code elimination mark the annotation type as unused

[TharmiganK]

Description

$Subject

Steps to Reproduce

Testing protobuf package with the dead code elimination and when I run the tests without any external dependency annotation. I am getting the following error:

        protobuf.types.any
[2024-11-13 17:16:37,531] SEVERE {b7a.log.crash} - $typedesce$MessageDescriptor 
java.lang.NoSuchFieldError: $typedesce$MessageDescriptor
        at ballerina.protobuf&0046types&0046any$test.1.tests.any_msg_pb.$annot_func$_911(tests/any_msg_pb.bal:25)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$gen$&0046&0060init&0062(protobuf.types.any:2)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$moduleInit(protobuf.types.any)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$moduleExecute(protobuf.types.any)
        at ballerina.protobuf&0046types&0046any$test.1.$_init.$lambda$$moduleExecute$(protobuf.types.any)
        at io.ballerina.runtime.internal.scheduling.SchedulerItem.execute(SchedulerItem.java:54)
        at io.ballerina.runtime.internal.scheduling.Scheduler.run(Scheduler.java:324)
        at io.ballerina.runtime.internal.scheduling.Scheduler.runSafely(Scheduler.java:291)
        at java.base/java.lang.Thread.run(Thread.java:840)

In the report, I see this type as a unused type:

    "unusedTypeDefNames": {
      "sourceTypeDefinitions": [
        "MessageDescriptor",
        "Error"
      ],
      "virtualTypeDefinitions": []
    },

This MessageDescriptor is the type of the annotation and the annotation is present in the test source code

Affected Version(s)

Ballerina SwanLake Update 11

OS, DB, other environment details and versions

Mac OS

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[MaryamZi]

Seems like this happens when the annotation is defined in the default module and a non-default module's tests use the annotation.

test_proj

public type Config record {|
    int value;
|};

public annotation Config annot on type;

public function main() {
    
}

test_proj.mod's tests

import ballerina/test;
import test_proj;

@test_proj:annot {
    value: 100
}
type TestType record {|

|};

@test:Config
function testFn() {
}

$ bal test --eliminate-dead-code --dead-code-elimination-report
Compiling source
        maryamm/test_proj:0.1.0
Duration for unused BIR node analysis : 1ms
    maryamm/test_proj:0.1.0 : 1ms


Running Tests

        test_proj.mod
[2024-11-14 13:26:57,174] SEVERE {b7a.log.crash} - $typedesce$Config 
java.lang.NoSuchFieldError: $typedesce$Config
        at maryamm.test_proj&0046mod$test.0.tests.mod_test.$annot_func$_816(tests/mod_test.bal:4)
        at maryamm.test_proj&0046mod$test.0.$_init.$gen$&0046&0060init&0062(test_proj.mod:2)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleInit(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$moduleExecute(test_proj.mod)
        at maryamm.test_proj&0046mod$test.0.$_init.$lambda$$moduleExecute$(test_proj.mod)
        at io.ballerina.runtime.internal.scheduling.SchedulerItem.execute(SchedulerItem.java:54)
        at io.ballerina.runtime.internal.scheduling.Scheduler.run(Scheduler.java:324)
        at io.ballerina.runtime.internal.scheduling.Scheduler.runSafely(Scheduler.java:291)
        at java.base/java.lang.Thread.run(Thread.java:840)

[MaryamZi]

In io.ballerina.projects.JBallerinaBackend#registerUnusedBIRNodes, the module gets sorted to the end, and then shouldOptimize(moduleContext) && (isRootModule(moduleContext) || moduleContext.isUsed()) evaluates to false for the module.

https://github.com/ballerina-platform/ballerina-lang/blob/feature-codegen-optimizer/compiler/ballerina-lang/src/main/java/io/ballerina/projects/JBallerinaBackend.java#L253

[Thushara-Piyasekara]

The reason for this behavior is because we added a constraint to not analyze non parent packages of testable packages during DCE. This was done to avoid false positive test results (see https://docs.google.com/document/d/1WNBN9hI7rbCL6wpozDAwImagUrM-sn7F/ for more details).

In the implementation, the logic for handling those scenarios was to stop the analysis if we refer to a non parent package node from a testable package.

Even though we stop the analysis, since the dependencies of testable packages are not optimized, the code will work. But in this case the logic used to determine testable pkg to build pkg usages fails.

We will have to update the logic to handle these cases or not support such cases.

I vaguely remember a having a conversation with @hasithaa about whether we should allow running tests from non build pkg modules with DCE flags. If I remember correctly the conclusion was not to allow such cases. But I cant remember the exact reasoning. I will look into this more and update.

[Thushara-Piyasekara]

In io.ballerina.projects.JBallerinaBackend#registerUnusedBIRNodes, the module gets sorted to the end, and then shouldOptimize(moduleContext) && (isRootModule(moduleContext) || moduleContext.isUsed()) evaluates to false for the module.

https://github.com/ballerina-platform/ballerina-lang/blob/feature-codegen-optimizer/compiler/ballerina-lang/src/main/java/io/ballerina/projects/JBallerinaBackend.java#L253

We can fix this by allowing the analyzer to enter the non default modules with testable pkgs by checking for moduleContext.bLangPackage().hasTestablePackage() condtion.

But this will not fully fix the issue because even though the analyzer goes inside the pkg it will stop analyzing the type reference because the following condition fails.

ballerina-lang/compiler/ballerina-lang/src/main/java/io/ballerina/projects/UsedTypeDefAnalyzer.java

Line 185 in 092cc69

!bType.tsymbol.pkgID.equals(usedBIRNodeAnalyzer.currentPkgID);

This condition only works for testable pkg's parent pkg references. We will have to introduce a new mechanism to determine build pkg references from non default testable pkgs.

Also we will have to update the logic in a similar manner for function invocations as well.

ballerina-lang/compiler/ballerina-lang/src/main/java/io/ballerina/projects/UsedBIRNodeAnalyzer.java

Line 471 in 092cc69

if (childPkgId.equals(currentPkgID)) {

I will update here if i find a proper way to do this.