A couple of concerns from this initial draft:

• flipping every single bit like this results in a significant slowdown of test_sig.c, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

• ideally we would test only signatures declared as SUF-CMA, should we add bool suf_cma the OQS_SIG struct? Or maybe replace bool euf_cma with an enum specifying the claimed security?

Tagging @dstebila and @SWilson4 for feedback on this.

A couple of concerns from this initial draft:

* flipping every single bit like this results in a significant slowdown of `test_sig.c`, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

I think it would suffice to select an index at random and flip that bit. In the event of a failure we'll want to output the bit index that triggered the error. Maybe we could make the number of iterations customizable via a command-line flag. That would allow us to do a minimal number of iterations in the standard CI tests but perhaps do more in the scheduled weekly runs.

* ideally we would test only signatures declared as SUF-CMA, should we add `bool suf_cma` the OQS_SIG struct? Or maybe replace `bool euf_cma` with an enum specifying the claimed security?

Fine by me to replace the boolean with an enum.

* flipping every single bit like this results in a significant slowdown of `test_sig.c`, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

Ouch. Before thinking thoroughly about, in mind signatures are short (few hundred bits) so it's "only" a factor of a few hundred, but of course PQ signatures are much longer, and so it's a factor of several thousand at least. Yeah, that's now a pain point. Sorry for setting you down the wrong track.

I think Spencer's suggestion of a command-line argument would be a good compromise.

* ideally we would test only signatures declared as SUF-CMA, should we add `bool suf_cma` the OQS_SIG struct? Or maybe replace `bool euf_cma` with an enum specifying the claimed security?

I think recording this in the OQS_SIG struct would make sense; for backwards compatibility, I'd go with introducing a new bool rather than changing the name/behaviour of the existing flag.