Correct-experimental-tests

[tijlleenders]

@kobe-reygel @thinkrapido
I've validated a few tests from the experimental folder.
Most of them have a backing in Excel - unless really too simple and/or annoying to make.

I haven't checked which ones are passing - but if any are - we can move them to the stable folder.

NB: The new budget test cases have a different json format than the current contract.
Let me know if you think that is appropriate - then we can make an issue to change the input expectation for all budget goals.

[kobe-reygel]

Out of the top of my head, only 'split-3' fails for the experimental testcases, so all the testcases that you validated should be good to move to 'stable'

I'll look at this PR somewhere next week!

[kobe-reygel]

@tijlleenders there are 2 new testcases that are failing.
I'll have a quick look at them. If I can't get them to succeed I'll keep them in a separate folder and introduce a mechanism of running tests in that folder (because as of now we only run tests in 'stable' and tests in 'experimental', the latter with the feature flag 'experimental-testset', so the tests in your 'validated' folder will never run)

[kobe-reygel]

NB: The new budget test cases have a different json format than the current contract.
Let me know if you think that is appropriate - then we can make an issue to change the input expectation for all budget goals.

oh I missed this, so I guess this is where the issue lies that those 2 do not succeed :D

[kobe-reygel]

@tijlleenders I see where you are coming from with the new budget json format. If we want to only support weekly and daily budgets for now I see nothing really wrong with that format.

But I also do not see a lot of advantage over our current format. The current format is more generic, extensible and flexible. I do not know if it's really worth it to put in the effort of changing the api (and hence also all the testcases) to make the budget api less generic.

What do you think? @thinkrapido

[kobe-reygel]

@tijlleenders I moved all passing testcases into stable.
I propose we keep the 2 remaining issues apart for now:

• budget-0-24-35-40 is blocked by the resolution of i#406
• budget-2-2-2-2 is not something that the scheduler knows how to deal with right now, I propose fleshing out what we want to do with such a type of input on a separate issue, and putting it on the backlog

[tijlleenders]

Ha, yeah just proposed the same!

[kobe-reygel]

I made the (currently pretty empty) issue #412 to deal with the budget-2-2-2-2 testcase.

I referenced the other testcase on #406, which should solve it.

I moved all other tests to stable, they are passing