Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bitflip test for trivial SUF-CMA forgeries #2090

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

rtjk
Copy link
Contributor

@rtjk rtjk commented Feb 27, 2025

This pull request adds a test against trivial SUF-CMA forgeries in signature schemes: for every bit of the signature, flip it and ensure that the verification fails.

As discussed in #1999.

  • Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)? (If so, a version bump will be required from x.y.z to x.(y+1).0.)
  • Does this PR change the list of algorithms available -- either adding, removing, or renaming? Does this PR otherwise change an API? (If so, PRs in fully supported downstream projects dependent on these, i.e., oqs-provider will also need to be ready for review and merge by the time this is merged.)

@rtjk
Copy link
Contributor Author

rtjk commented Feb 27, 2025

A couple of concerns from this initial draft:

  • flipping every single bit like this results in a significant slowdown of test_sig.c, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

  • ideally we would test only signatures declared as SUF-CMA, should we add bool suf_cma the OQS_SIG struct? Or maybe replace bool euf_cma with an enum specifying the claimed security?

Tagging @dstebila and @SWilson4 for feedback on this.

@SWilson4
Copy link
Member

A couple of concerns from this initial draft:

* flipping every single bit like this results in a significant slowdown of `test_sig.c`, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

I think it would suffice to select an index at random and flip that bit. In the event of a failure we'll want to output the bit index that triggered the error. Maybe we could make the number of iterations customizable via a command-line flag. That would allow us to do a minimal number of iterations in the standard CI tests but perhaps do more in the scheduled weekly runs.

* ideally we would test only signatures declared as SUF-CMA, should we add `bool suf_cma` the OQS_SIG struct? Or maybe replace `bool euf_cma` with an enum specifying the claimed security?

Fine by me to replace the boolean with an enum.

@dstebila
Copy link
Member

* flipping every single bit like this results in a significant slowdown of `test_sig.c`, on my Skylake laptop it went from basically instantaneous to taking 2 minutes for a single signature (tested with cross-rsdp-128-balanced)

Ouch. Before thinking thoroughly about, in mind signatures are short (few hundred bits) so it's "only" a factor of a few hundred, but of course PQ signatures are much longer, and so it's a factor of several thousand at least. Yeah, that's now a pain point. Sorry for setting you down the wrong track.

I think Spencer's suggestion of a command-line argument would be a good compromise.

* ideally we would test only signatures declared as SUF-CMA, should we add `bool suf_cma` the OQS_SIG struct? Or maybe replace `bool euf_cma` with an enum specifying the claimed security?

I think recording this in the OQS_SIG struct would make sense; for backwards compatibility, I'd go with introducing a new bool rather than changing the name/behaviour of the existing flag.

@SWilson4 SWilson4 linked an issue Feb 27, 2025 that may be closed by this pull request
rtjk added 4 commits March 4, 2025 08:31
Signed-off-by: rtjk <[email protected]>

Signed-off-by: rtjk <[email protected]>
Signed-off-by: rtjk <[email protected]>

Signed-off-by: rtjk <[email protected]>
- add command-line argument to test_sig.c (the number of bitflips)
- update CROSS upstream to SUF-CMA
Signed-off-by: rtjk <[email protected]>

Signed-off-by: rtjk <[email protected]>
@rtjk rtjk force-pushed the add-bitflip-test branch from 9bf3def to e937e3b Compare March 4, 2025 07:32
@rtjk rtjk force-pushed the add-bitflip-test branch from 23091fd to b0e32ba Compare March 4, 2025 08:17
Comment on lines +371 to +376
# assume EUF-CMA for schemes that don't specify a security classification
scheme['metadata']['euf_cma'] = 'true'
scheme['metadata']['suf_cma'] = 'false'
if 'claimed-security' in metadata:
if metadata['claimed-security'] == "SUF-CMA":
scheme['metadata']['suf_cma'] = 'true'
Copy link
Contributor Author

@rtjk rtjk Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a recap of the claimed security of signature schemes as of now:
-ML-DSA is declared as SUF-CMA in the liboqs docs, but not in the upstream META.yml
-CROSS is declared as SUF-CMA both in the docs and in META.yml
-all other signatures are declared as EUF-CMA in the docs, but make no claim in META.yml
-stateful signatures don't report any claimed-security

From here, I think it should be clarified who has the ultimate responsibility for making the security claim: the liboqs docs or the upstream repo? The latter seems preferable, as it is what is done for KEMs.

If we choose the liboqs docs instead, then it would be necessary to edit sig->suf_cma based on the documentation (e.g., the claimed-security field in docs/algorithms/sig/ml_dsa.yml).

Also, is it okay to assume EUF-CMA for imported schemes? This was the previous behavior, which I have kept.

Finally, how should we treat stateful signatures? Should I add the bitflip test there too?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Strengthen signature verification failure tests
3 participants