Feature request: Monkey tests in separate project

[moaz-mokhtar]

I would like to propose a new feature that I believe would greatly enhance the functionality of our project. I suggest that we add a test generator which would automatically create or randomize test JSON files.

The main reason for my proposal is to minimize the effort that goes into creating test files for each case. As we know, our system provides complex output, and it takes a significant amount of time and effort to create test JSON files input,json and output.json.

To support my proposal, I can provide an example of a similar functionality that I have previously implemented. I created functions for each struct to generate objects with constrained random values, which proved to be a valuable addition to the project.

This monkey test generator is best done in a separate project - which uses the ZinZen-scheduler crate lib.

[tijlleenders]

The input can already be 'generated' by the front end. Setting the test case values is not random but manual.
Is this sufficient to speed up your current approach?

The output can be generated by running this input through the scheduler - but then requires a manual check to verify if it corresponds to expectation.

How do you want to generate expected output without 'rebuilding' the scheduler?

[moaz-mokhtar]

• It is not mandatory for follow this now or at the current stage. We brainstorm, design, and decide.
• I found a need for this idea to minimize our efforts and provide automated randomized tests to cover more cases.

---

I don't have in mind a design now for this, but have the basic idea and inspiration. Inspiration is from functionalities which I did develop before and I can summarize it for brain storming.

// # Imagine you have struct with insert method
struct User{
    id: String,
    name: String,
    age: i32,
}


fn insert(user: User) -> User{
    //...
    // Insert user and return it from DB
}


// # Now we want to create a test for insert method
fn test_user_insert(){
    let expected: User = User{
        id: "USR11-02-21320".to_string(),
        name: "Test User 1",
        age: 23
    };

    let added = insert(expected);

    assert_eq!(expected.id, added.id);
    //...
}

Now after developing a basic generator for User struct, we can use it as below:

// # Function to generate a randomized data for User
impl User{
    fn mock(count:i32) -> Vec<User> {
        // ...
	let mut users = vec![];
        
        for _i in count{
            let id = random_id_value(); // Random id
            let name = random_person_value(); // Random person's name
            let age = random_value(0, 90); // Random value up to 90 to reprsent age
			
            users.push(User{
                id,
                name,
                age
            });
        }
        
        users
    }
}


// # Use generators in test cases to minimize efforts to generate object and covering various cases
fn test_user_insert(){
    let expected: User = User::mock(1);

    let added = insert(expected);

    assert_eq!(expected.id, added.id);
}

So we can utilize and improve like this functionality and find a good design which will leads to:

• generating goals for tests or similar with countless variations
• can develop criteria like User::mock(count)

[moaz-mokhtar]

I have sample repo which illustrates basic idea:
https://github.com/moaz-mokhtar/mock-generator

[tijlleenders]

Since we can't generate the correct/expected output, the value is only in generating random valid input to see if we can find corner cases where the scheduler breaks when it shouldn't.

Finding corner cases is not a prio for the project right now.

[tijlleenders]

After reading https://internals.rust-lang.org/t/pre-rfc-dynamic-tests/6790 and https://internals.rust-lang.org/t/dynamic-tests-revisited/18095 ... this 'generator' should be a separate rust project that uses the ZinZen lib crate.

[moaz-mokhtar]

Yes maybe, but initially, I'm developing little generators gradually to minimize efforts to make unit tests for the functions.

[tijlleenders]

@moaz-mokhtar Yeah that is great.
The 'little' generators are useful and since they are static code - no problem at all to include them as utils.
I've renamed this issue to monkey test.

[tijlleenders]

@moaz-mokhtar the rstest crate might be useful for this.

[moaz-mokhtar]

@moaz-mokhtar the rstest crate might be useful for this.

I did a trial with it, basically for primary values it is good, but for complex objects it is easier for me this time to implement mocks_generator which mentioned before. I created a simple module mocking which have little useful functions needed to mock data for tests or similar. Check commit: 0cad514