You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At first I thought this was a bug in hspec, I opened an issue at hspec/hspec#899. But as things currently stand I'm not sure how hspec could fix it without changes to quickcheck. So I'm posting here too to try to get more eyes.
Short version is that hspec has an around function that lets you wrap every run of a quickcheck test. But if the test fails and shrinks, it doesn't wrap the shrunk attempts. For example:
before
Testing 0
after
before
Testing 1
after
before
Testing 2
after
Testing 1
Testing 0
We have two successful tests that get wrapped, a failing test that gets wrapped, then two shrinks of the failing test that don't get wrapped.
The way this works is that hspec examples can be of type a -> Property, and hooks can provide the a. a -> Property is essentially a -> QCGen -> Int -> Rose Result, where the root Result is the result of the first test and children are the results of any shrinks. (Even if shrinking is unnecessary, every shrink is in the tree, though not actually evaluated.) But that type can't do what we want, since we have a single a to construct the whole tree. Currently it essentially calls hook $ \a -> reduceRose (r a), which means the function passed to the hook
Does the IO necessary to get the root Result of the tree
Does the IO necessary to get the IO necessary to get any immediate child Results of the tree
Does not get any child Results of the tree.
Which gives us the behavior observed, that the hook wraps the initial test but not any shrinks.
I don't see any way to get around this with the types as they are. Switching to QCGen -> Int -> Rose (a -> IO Result) might work, but it feels like that would need large changes to both quickcheck and hspec, and I can't rule out that it would fail for some other reason.
Any suggestions?
The text was updated successfully, but these errors were encountered:
At first I thought this was a bug in hspec, I opened an issue at hspec/hspec#899. But as things currently stand I'm not sure how hspec could fix it without changes to quickcheck. So I'm posting here too to try to get more eyes.
Short version is that
hspec
has anaround
function that lets you wrap every run of a quickcheck test. But if the test fails and shrinks, it doesn't wrap the shrunk attempts. For example:We have two successful tests that get wrapped, a failing test that gets wrapped, then two shrinks of the failing test that don't get wrapped.
The way this works is that hspec examples can be of type
a -> Property
, and hooks can provide thea
.a -> Property
is essentiallya -> QCGen -> Int -> Rose Result
, where the rootResult
is the result of the first test and children are the results of any shrinks. (Even if shrinking is unnecessary, every shrink is in the tree, though not actually evaluated.) But that type can't do what we want, since we have a singlea
to construct the whole tree. Currently it essentially callshook $ \a -> reduceRose (r a)
, which means the function passed to the hookResult
of the treeResult
s of the treeResult
s of the tree.Which gives us the behavior observed, that the hook wraps the initial test but not any shrinks.
I don't see any way to get around this with the types as they are. Switching to
QCGen -> Int -> Rose (a -> IO Result)
might work, but it feels like that would need large changes to both quickcheck and hspec, and I can't rule out that it would fail for some other reason.Any suggestions?
The text was updated successfully, but these errors were encountered: