Chatty checkers

Chatty checkers provide extra information about a failure. There are three ways to create them. Two depend on a checker function having the right "shape". The other is less convenient, but more general.

Chatty results from boolean combinations

Suppose you want to check integer results to see if they're not only primes, but specifically Mersenne primes. Mersenne primes are both Marsenne numbers and primes. You could write such a checker this way:

(defn mersenne-prime [actual]
  (and (= prime? actual)
       (mersenne-number? actual)))

Here's a false fact and its failure report:

user=> (fact 131071 => mersenne-prime)

FAIL at (t_bug.clj:27)
Actual result did not agree with the checking function.
        Actual result: 131071
    Checking function: mersenne-prime

That's not hugely helpful. Did it fail because it's not prime or because it's not Mersenne? To write a more expressive checker, use combining checkers:

user=> (def mersenne-prime (every-checker prime? mersenne-number?))
user=> (fact 131071 => mersenne-prime)

FAIL at (NO_SOURCE_FILE:2)
Actual result did not agree with the checking function.
        Actual result: 131071
    Checking function: mersenne-prime?
    During checking, these intermediate values were seen:
       mersenne-number? => false
false

Chatty results from subcalculations

Generous angel investors our funding our Big Data startup. Key to its success is the ability to crunch data so that the output, when "lateralized" produces a descending sequence of values. Such output is referred to as "nonelian". Put in concrete terms, success depends on implementing code such that facts like the following check out:

(fact (crunch :by-county :gauss 5.0) => nonelian)

Here's the definition of the nonelian checker:

(defn nonelian [actual] (apply > (lateralize actual)))

And here's the result of a check:

user=> (fact (crunch :by-county :gauss 5.0) => nonelian)

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: nonelian
false

This isn't horrible information, but it would also be nice to see the intermediate results of lateralize. Fortunately, that's easy to do by using chatty-checker in place of fn:

user=> (def nonelian (chatty-checker [actual] (apply > (lateralize actual))))
user=> (fact (crunch :by-county :gauss 5.0) => nonelian)

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: nonelian
    During checking, these intermediate values were seen:
       (lateralize actual) => [5.0 4.98 4.63 4.62 5.0 4.3]
false

chatty-checker converts each top-level function application in its body into a form that reports its results.

It's equally easy to use chatty-checker to make checkers that take arguments. Let's change "nonelian" so that there are variants. The one you've already seen is "Strictly down nonelian" (because it uses >). "Strictly up nonelian" uses <, etc. So here's a checker that's parameterized by direction:

user=> (defn nonelian [comparison]
         (chatty-checker [actual] (apply comparison (lateralize actual))))
user=> (fact (crunch :by-county :gauss 5.0) => (nonelian >))

FAIL at (NO_SOURCE_FILE:1)
Actual result did not agree with the checking function.
        Actual result: [1.7 8.32 2.0 8.3 0.0 0.0 12.01]
    Checking function: (nonelian >)
    During checking, these intermediate values were seen:
       (lateralize actual) => [5.0 4.98 4.63 4.62 5.0 4.3]
false