HI-03 Insucient Timeout Handling Could Lead to Denial of Service

[xoloki]

The FIRE (meta)protocol makes provisions to remove malicious actors and to restart rounds
(or sessions) in certain instances, including for example, when time passess without
protocol state changes. However, checks are insufficient and bad actors could stall the
protocol ignoring these precautions.
For example, the process_inbound_messages() function in the FIRE coordinator seems to
be susceptible to a DOS. The function includes a first block of code where input messages
are processed, and a second block of code that processes results from the previous
codeblock results and takes care of timeouts. If for some reason the stalling were to occur in
the first block, the logic in this method could never exit and the stalling condition would
persist.
One way to exploit this consists in a signer sending a huge amount of shares/nonces for the
correct dkg_id. All of these messages will be processed and this may take more time than
what the timeout allows, yet the time check will happen much later in the code causing
fast-forwarding or failure of a DKG or signing round.

[xoloki]

Similarly with CR-01/02 (#66 #67) sbtc-signer does not make use of the timeouts, since it has its own timeouts layered above. A number of DOS issues were discovered and fixed during the launch period, which mitigates this for other users of the library.

It is likely that in the future, we will simply remove fn process_inbound_messages entirely. It makes not only the timeouts difficult to handle properly, but also passing Err returns up the stack when processing multiple messages. sbtc-signer already bypasses it, and there were discussions about doing so in the stacks-signer dev process as well. Then we can process timeouts directly before or after any message without the long blocking queue problem.