Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot establish network connection between two U280s with default shell #70

Open
lifeis-hard opened this issue Jan 10, 2025 · 0 comments

Comments

@lifeis-hard
Copy link

lifeis-hard commented Jan 10, 2025

Hi.
Thank you all for this amazing project.

I am trying to establish a 100G network connection with two U280s (only using single port for each board).
I built the default open-nic shell with command below. I am using Vivado version 2022.2.

vivado -mode batch -source build.tcl -tclargs -board au280 -impl 1 -post_impl 1 -num_phys_func 2 -num_cmac_port 2 -num_qdma 1 -user_plugin ../plugin/p2p/

The build finished with some amount of timing errors as below.

WNS: -0.256ns
TNS: -204.714ns

I just tried to program the bitstream to two U280s since I just wanted to check a simple network connection without any custom design implemented (I plugged in the default p2p).

After reboot, the open-nic driver successfully captured the boards. Command lspci -vd 10ee: showed expected results.
The driver was built with RS_FEC disabled. (I manually disabled from onic_main.c)

I physically connected each port 0 of U280 with QSFP28-QSFP28 DAC cable directly.
However, the ping test did not go well.

The ping test was held as below:

  1. In both servers, each containing a single U280 built with default open-nic shell, the driver assigned enp1s0f0 as master PF interface.
  2. I assigned IP address 192.168.1.10/24 and 192.168.1.20/24 to each enp1s0f0 interface.
  3. I left the 0x8090 register value to zero. However, the 0x8204 showed 0xC0, which seems not fine.
  4. I tried to ping in both directions, but neither of the directions successfully sent the pings.

I checked that the loopback register is working pretty much well.
When I wrote 0x1 to 0x8090 and sent ping again, I found that ARP packets are duplicating with tcpdump. Still, there were no other packets detected.

Also, I noticed that I could receive 0x03 from 0x8204 register only when I enabled the loopback (Wrote 0x1 to 0x8090).
Additionally, the ethtool command showed Link detected: yes only when the loopback was enabled.
If the 0x8090 value was set to zero, both 0x8204 register value and ethtool information showed negative connection.

Is it able to establish board-to-board connection, with just default shell design?
If then, I wonder what am I doing wrong.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant