nDPI payload length checks and MTU

[utoni]

Some protocol dissectors (especially the TCP ones) are using length checks from dissected payload and comparing it with payload_packet_len to successfully detect (or not) a certain protocol. As it is effective, because there are low chances for false-positives within other protocols, it may also dependent on the MTU size of the network interface.

For example: The ElasticSearch dissector uses that detection pattern. But a single ES message can easily exceed the MTU on the network interface nDPI is running on, because it is split into smaller fragments that do not exceed the MTU, causing a false-negative detection.

As by now nDPI does not provide any support to reassemble TCP fragments.
But IMHO we could at least extend nDPI to provide two new API functions e.g.:

ndpi_set_mtu(ndpi_struct, mtu); // set MTU allowed for this nDPI instance
ndpi_check_packet_length(ndpi_struct, packet, dissected_length); // returns 1 iff packet->payload_packet_len matches dissected_length OR dissected_length is greater than MTU, 0 otherwise

Downside: This may increase false-positive rate.
Does this make sense? What do you guys think?

(can not share the ES PCAP as it contains data I do not own)

[mmaatuq]

• I think it's useful to have such apis as i had a similar experience with a private protocol
• some concerns:
  • how to set a reliable value for MTU ( not so sure if the MTU of capture interface is enough)
  • should only used by a specific dissector (dissectors where we know what we can do with next data chunk)
  • what MAX dissected len should be allowed as it could be corrupted (maybe cap for jumbo frame 9k)