SRC port and DST port number are switched sometimes in ntopng

[ioesoft]

Environment:

• OS name: Ubuntu
• OS version: 20.04.2 LTS
• Architecture: x86_64
• ntopng version/revision: 6.2.250103

What happened:
In ntopng, when checking the flow, there are occasional cases where the source port and target port are swapped. For example, in the case of port 443, which should typically be considered the target port, there are flows that are sometimes displayed as the source port. I previously reported this bug, and Matteo mentioned that it was fixed in version 6.0, but the issue still persists in the latest version.
Regarding the issue mentioned above,
1) Could it be a problem with nprobe, or is it a problem with ntopng?

How did you reproduce it?
FYI, here is my configuration of ntop products:
nProbe(export over ZMQ) --> ntopng(export over Syslog) --> Elastic Search

In my environment, nprobe sends flows data to ntopng via ZMQ. The data volume is very high, and due to limited bandwidth between nprobe and ntopng, a large number of ZMQ drops are occurring.
2) Could these ZMQ drops be the cause of the issue mentioned above?

Debug Information:

[lucaderi]

Can you please show the TCP flags per direction (cli->srv and src->cli) of the abnormal flows. Please also report the nprobe configuration for -T

[ioesoft]

Here is the configuraion for -T:
-T="@NTOPNG@ %JA3C_HASH %JA3S_HASH %SRC_AS %DST_AS %SRC_AS_MAP %DST_AS_MAP %MAX_IP_PKT_LEN %ICMP_TYPE %FLOW_END_REASON %APPL_LATENCY_MS %L7_PROTO_RISK %L7_PROTO_RISK_NAME %L7_RISK_SCORE %FLOW_VERDICT %L7_RISK_INFO %SMTP_MAIL_FROM %SMTP_RCPT_TO %HTTP_X_FORWARDED_FOR %CLIENT_TCP_FLAGS %SERVER_TCP_FLAGS"

And, I will also show the TCP flags per direction once I get it from my client.

[lucaderi]

Can you please show the TCP flags per direction (cli->srv and src->cli) of the abnormal flows. Please also report the nprobe configuration for -T

Please show the TCP flags per direction (cli->srv and src->cli) of the abnormal flows. Paste a picture showing one of those flows with wrong direction, to understand the problem.

[lucaderi]

@ioesoft Any news?

[ioesoft]

This issue occurs very intermittently, so it is almost impossible to monitor it continuously until the issue arises. As you know, in ntopng, flows are deleted quickly (within 2 minutes), so unless we are lucky enough to capture it when the issue occurs, it is very difficult to capture the event.

The best approach is to retrieve data from ELK, as ntopng sends all flows to ELK for storage, and we can query the data stored in ELK. The result is the screen capture I sent in previous comment above.

For now, we are waiting for a response from the client regarding the TCP flags for both directions.

So, I was wondering if we could find a solution by analyzing the source code for any potential causes.

The customer is planning to purchase 30-40 new nprobes and renew around 200 existing nprobe AMS licenses by the next month. However, due to this issue, the customer has concerns about the stability and data integrity of the NTOP product and has expressed dissatisfaction.

Again, is there any way to find a solution by analyzing the source code for any potential causes?

[lucaderi]

very likely the problem is created by TLS but I need to see evidence of the problem in order to find the root cause. I think the DB has enough info to troubleshoot this problem without supervising the web GUI. Please check if you can export some data that might help with troubleshooting.

[ioesoft]

Thanks. I will check with the client if they can provide the database information and let you know. However, if you look at the snapshot above I provided earlier, you can also see that swap were happened not only on port 443 but also on port 80.

[lucaderi]

sure but i need to check flags, timing, bytes etc so please provide the requested data