Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API for the general-purpose model checker #411

Open
wants to merge 12 commits into
base: develop
Choose a base branch
from

Conversation

dmitrii-artuhov
Copy link
Collaborator

@dmitrii-artuhov dmitrii-artuhov commented Oct 7, 2024

Description:

PR adds function for running lincheck model checker algorithm on arbitrary code via lambda body.
Usage example:

class JunitTest {
  val invocations = 100
  @Test
  fun test() = runConcurrentTest(invocations) {
    var counter = 0
    val t1 = thread { counter++ }
    val t2 = thread { counter++ }
    t1.join(); t2.join()
    check(counter == 2) // will detect an error an throw an exception
  }
}

Additional improvements to the PR:

@dmitrii-artuhov dmitrii-artuhov requested a review from eupp October 7, 2024 14:04
@dmitrii-artuhov dmitrii-artuhov self-assigned this Oct 7, 2024
@dmitrii-artuhov dmitrii-artuhov marked this pull request as draft October 7, 2024 14:04
@eupp
Copy link
Collaborator

eupp commented Oct 7, 2024

@dmitrii-artuhov thanks for the PR.
Could you please repair the CI before we start reviewing this.

@dmitrii-artuhov dmitrii-artuhov changed the title [Draft] Lincheck new API function Lincheck new API function Oct 17, 2024
@ndkoval ndkoval changed the title Lincheck new API function API for the general-purpose model checker Nov 25, 2024
@dmitrii-artuhov dmitrii-artuhov marked this pull request as ready for review January 14, 2025 12:46
@dmitrii-artuhov dmitrii-artuhov requested a review from eupp January 14, 2025 13:01
* @param block lambda which body will be a target for the interleavings exploration.
*/
@JvmStatic
fun <R> verifyWithModelChecker(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @ndkoval .

For now, lets rename this function into runConcurrentTest, and make it top-level function (no Lincheck object needed). This is not a final name, we will discuss API of this function (including its name) in more detail before public release of this feature.

}
failure.checkLincheckOutput(outputFileName)
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are also tests in CustomThreadsRepresentationTest.kt file that I have added when I was working on supporting custom threads. Could you please move them in this file, rewrite them to use verifyWithModelChecker function and check that they would still work?

@dmitrii-artuhov dmitrii-artuhov requested a review from eupp January 16, 2025 16:51
@dmitrii-artuhov
Copy link
Collaborator Author

dmitrii-artuhov commented Jan 16, 2025

@eupp I added the testClassInstance parameter to the runConcurrentTest function, because right now I utilize a specifal wrapper class which calls the lambda and only this wrapper is recreated between iterations, not the user class which might contain fields that are modified. Since on trace collection there are in total 2 iterations, then I need to restore the fields of the actual user test class. Required functionality was added to the SnapshotTracker class, so that it restores the user test class instance between iterations.

@eupp
Copy link
Collaborator

eupp commented Jan 16, 2025

I added the testClassInstance parameter to the runConcurrentTest function

Let's please revert this change. The API should be as simple as it could be, that is the point why we adding it. It should only take the lambda to run and that's it.

I utilize a specifal wrapper class which calls the lambda and only this wrapper is recreated between iterations, not the user class which might contain fields that are modified.

To make the test work with only the lambda, please remove any state of the test class by either:

  • making the respective fields static, or
  • promoting the fields to regular variables inside lambda.

Use one method or another, depending on which makes more sense for a particular test.

This way we will eliminate any "external" state outside lambda (except static state, but it will be handled by the snapshot tracker).

In general, there is indeed a problem here --- the problem is that the lambda that we pass to the model checker should not capture any state, in order to be deterministically reproducible.
I will add this problem to the design document, and we will discuss it and possible solutions separately.

One of possible solutions is similar to what you propose --- to force the user to explicitly mention all captured objects as method parameters. But we need to discuss this solution and how to express it in the API in more detail before implementing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants