Delay using client hints for TCP until 1st data packet

[ldir-EDB0]

I have a Bittorrent peer that tries to behave nicely in that it marks packets as LE (Least Effort) however it has found a corner case that needs some thought.

The scenario was that I was downloading a torrent and the packets leaving me were correctly marked 'LE' and going into the correct CAKE tin however the reply packets were being marked as 'CS0'. Both top bits were set, so connection has gone through static classify and not been caught and is now being handled by dynamic classify, yet leaving packets are 'LE'. How can this be?

In this particular case the flows were TCP flows and so had the usual TCP ephemeral port numbers, hence not being caught by static classify and not really catchable by static since there may be other traffic in those ranges. But these packets have an LE DSCP so should have been caught by the 'client hints'.

Wireshark showed me the initial TCP 3-way handshake is done at CS0 and LE is used for the data. Static classify was using the value from the connection open sequence. My horrible workaround to this is the following rule as the penultimate rule in static classify, in essence, for tcp connections with CS0 DSCP wait until 3 packets have passed before going through the rules.

#Wait for 1st tcp data packet before accepting classification
meta l4proto tcp ip dscp cs0 ct original packets < 3 counter return
meta l4proto tcp ip6 dscp cs0 ct original packets < 3 counter return

Set the dynamic conntrack bit on unclassified connections

ct mark set ct mark & $ct_unused | $ct_dynamic counter

Thoughts?

[jeverley]

That's a really interesting scenario, I'll have a think about how best to address this later this evening.

Am I correct in understanding that it's the next client originating packet after the 3 way handshake that contains the LE class?

[ldir-EDB0]

Local is my LAN based bittorrent peer, remote is the remote Internet WAN based peer.

Local>Remote: SYN CS0
Remote>Local: SYN,ACK CS0
Local>Remote: ACK, CS0 - (ct state is established at this point so I can't used established/not established as the decision point, I need the packet after)
Local>Remote: PSH,ACK LE

[jeverley]

In 773eb32 I've tried to avoid tying us to a particular number of packets, instead opting to allow detection of client DSCP marks at point in time whilst the connection has the dynamic mark (which is until it's been marked with a specific DSCP CT value).

I do suspect though that it might result in your connections falling into the threaded P2P client detection and being marked as LE before it even specifies its own mark (not ideal since we might have some clients that genuinely need a higher class).

I'm experimenting with adding to the post routing/input chains as its own dedicated jump that could override the dynamic classification - to do so I'll need to preserve the dynamic bit which we currently clear in the ct verdicts.
I feel like this might be a better approach as it would allow a client hint to override a dynamic class at any time.

[jeverley]

Hi @ldir-EDB0 I've updated the dev branch with the following changes to address the above.

• We now only jump to dynamic_classify if both the ct_dynamic bit is set and ct_dscp == 0 (previously we just checked ct_dynamic).
• Dynamic classification verdicts (threaded clients/services) preserve the ct_dynamic bit.
• Client hints will be applied to any connection with ct_dynamic set regardless as to whether it has previously been DSCP classified.
• Users now have the ability to disable the threaded client/service dynamic connection classification modes.

[ldir-EDB0]

I shall give that a test soon.