Enable IPv6 #188

rbo · 2024-07-02T11:58:48Z

RH internal IT already provided an IPv6 Subnet

The IPv6 network will be from 2620:52:0::/48 and that is our internal only IPv4 mapped space.
There is no external IPv6 connectivity possible with that IP block.
Configuration:
Address assignment method: SLAAC
Net: 2620:52:0:2060::/64
Default GW: 2620:52:0:2060::ffe (but irrelevant as that information is sent/received via RA)

IPv6 VPN available at IAD2 and BRQ2

Tasks

Enable IPv6 / Add static IPv6 at inf1 & inf31 for DNS
Provide DNS via DHCPv6
Move DHCPv6 from inf49 to inf1

rbo · 2024-08-07T07:16:11Z

https://www.ipv6forum.com/dl/presentations/IPv6-addressing-plan-howto.pdf#page7

rbo · 2024-08-07T14:53:37Z

https://www.networkacademy.io/ccna/ipv6/stateless-address-autoconfiguration-slaac

[root@inf49 ~]# ip a
6: ens2f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master coe0 state UP group default qlen 1000
    link/ether 00:1b:21:b5:6a:20 brd ff:ff:ff:ff:ff:ff
    altname enp11s0f0
qlen 1000
    link/ether 00:1b:21:b5:6a:20 brd ff:ff:ff:ff:ff:ff
    inet 10.32.96.49/20 brd 10.32.111.255 scope global dynamic noprefixroute coe0
       valid_lft 7309sec preferred_lft 7309sec
    inet6 2620:52:0:2060:4f17:b5b3:522:5ce2/64 scope global dynamic noprefixroute
       valid_lft 2591997sec preferred_lft 604797sec
    inet6 fe80::9bfe:b2b1:b2f8:e8d7/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
[root@inf49 ~]# ipcalc 2620:52:0:2060:4f17:b5b3:522:5ce2/64
Full Address:	2620:0052:0000:2060:4f17:b5b3:0522:5ce2
Address:	2620:52:0:2060:4f17:b5b3:522:5ce2
Full Network:	2620:0052:0000:2060:0000:0000:0000:0000/64
Network:	2620:52:0:2060::/64
Netmask:	ffff:ffff:ffff:ffff:: = 64

Address space:	Global Unicast
HostMin:	2620:52:0:2060::
HostMax:	2620:52:0:2060:ffff:ffff:ffff:ffff
Hosts/Net:	2^(64) = 18446744073709551616
[root@inf49 ~]#

Mac	IPv6
`00:1b:21:b5:6a:20`	`2620:52:0:2060:4f17:b5b3:522:5ce2/64`

¯\_(ツ)_/¯

I don't know how SLAAC & RHEL9 works... and how rhel determinate the IPv6 Address.

rbo · 2024-08-08T19:16:46Z

Static IPv6 addresses for DNS:

2620:52:0:2060::53:1 => inf1
2620:52:0:2060::53:2 => inf31

rbo · 2024-08-08T19:53:55Z

Enabled dhcpv6 at inf1 and configured:

#
# DHCPv6 Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd6.conf.example
#   see dhcpd.conf(5) man page
#

option dhcp6.info-refresh-time 21600;
subnet6 2620:52:0:2060::/64
{
    option dhcp6.domain-search "coe.muc.redhat.com";
    option dhcp6.name-servers 2620:52:0:2060::53:1, 2620:52:0:2060::53:2;
    interface eno1;
}

But a the RA the O-Flag is 0 so client don't ask DHCPv6 for DNS:

ICMP Fields:

      Type           134

      Code           0

      Checksum       The ICMP checksum.  See [[ICMPv6](https://datatracker.ietf.org/doc/html/rfc4861#ref-ICMPv6)].

      Cur Hop Limit  8-bit unsigned integer.  The default value that
                     should be placed in the Hop Count field of the IP
                     header for outgoing IP packets.  A value of zero
                     means unspecified (by this router).

      M              1-bit "Managed address configuration" flag.  When
                     set, it indicates that addresses are available via
                     Dynamic Host Configuration Protocol [[DHCPv6](https://datatracker.ietf.org/doc/html/rfc4861#ref-DHCPv6)].

                     If the M flag is set, the O flag is redundant and
                     can be ignored because DHCPv6 will return all
                     available configuration information.

      O              1-bit "Other configuration" flag.  When set, it
                     indicates that other configuration information is
                     available via DHCPv6.  Examples of such information
                     are DNS-related information or information on other
                     servers within the network.

https://datatracker.ietf.org/doc/html/rfc4861

rbo · 2024-08-09T07:24:33Z

Created a ticket, to update RA configuration: https://redhat.service-now.com/help?id=rh_ticket&table=sc_req_item&sys_id=0be58f5b9773061cd658b82bf253af6c

rbo · 2024-08-16T14:20:51Z

RA is adjusted.

We provide IPv6 addresses via our DHCP server.

rbo · 2024-08-16T15:56:48Z

Now the Yes the RA looks different:

Managed address configuration is now SET
autonomous address-configuration flag is NOT SET anymore.

I don't get an IPv6 default gateway anymore.

And I'm not able to provide an IPv6 via DHCPv6, I see a lot of bad udp cksum related to the dhcpv6:

17:47:48.709929 IP6 (flowlabel 0xd45de, hlim 1, next-header UDP (17) payload length: 104) fe80::2e76:8aff:fe4f:61a4.dhcpv6-client > ff02::1:2.dhcpv6-server: [bad udp cksum 0x176a -> 0x49b9!] dhcp6 solicit (xid=6b42f0 (rapid-commit) (IA_NA IAID:3055685611 T1:0 T2:0) (Client-FQDN) (option-request DNS-server DNS-search-list SNTP-servers NTP-server opt_82) (client-ID type 4) (elapsed-time 65535))

rbo · 2024-08-21T15:18:22Z

Looks like inf1 (connected to arista-rj45) does not receive any DHCPv6 traffic.
My test VM running on ucs56 (connected to arista-rj45) sent DHCPv6 request and see a lot of DHCPv6 noise.
For switch config paraty I disabled ip dhcp relay always-on on arista-rj45

Inf1 uptime is 202 days since that no update and reboot. Let's update and reboot inf1 and inf31 tomorrow morning around 10 CEST

COE users informed via slack

rbo · 2024-08-22T09:37:51Z

inf1 and inf31 are connect to Red Hat Switch
All other Hosts are connected to Lab switches

Asked RH NoC to check the switch config.

rbo · 2024-08-22T12:24:17Z

Installed a dhcpd6 on inf49. Inf49 becomes all Solicit messages. But did not responde with an Advertise.

Source: https://www.alliedtelesis.com/sites/default/files/documents/configuration-guides/dhcpv6_feature_overview_guide.pdf

Good overview: https://blog.marquis.co/posts/2015-12-22-configuring-a-dual-stacked-dhcp-server/

rbo · 2024-08-22T12:32:24Z

[root@rhel9-harlequin-kite-11 ~]# sudo dhclient -6 -v eth0
Internet Systems Consortium DHCP Client 4.4.2b1
Copyright 2004-2019 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on Socket/eth0
Sending on   Socket/eth0
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 1050ms.
XMT: Forming Solicit, 1050 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 2110ms.
XMT: Forming Solicit, 3170 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Request renew in  +3600

rbo · 2024-08-22T12:50:27Z

UDP packages was not getting to dhcpd6:

[root@inf49 ~]# lsof  -i :547
COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
dhcpd   4418 dhcpd    5u  IPv6  41530      0t0  UDP *:dhcpv6-server
[root@inf49 ~]# strace -ff -e trace=network -s 10000  -p 4418

firewalld was ne problem at inf49... the temp host

rbo · 2024-08-22T12:51:29Z

[root@rhel9-harlequin-kite-11 ~]# sudo dhclient -6 -v eth0
Internet Systems Consortium DHCP Client 4.4.2b1
Copyright 2004-2019 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on Socket/eth0
Sending on   Socket/eth0
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 1060ms.
RCV: Advertise message on eth0 from fe80::8c4:60ba:1abe:3b64.
RCV:  X-- IA_NA 6d:00:00:01
RCV:  | X-- starts 1724330792
RCV:  | X-- t1 - renew  +0
RCV:  | X-- t2 - rebind +0
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
RCV:  | | | X-- Preferred lifetime 54000.
RCV:  | | | X-- Max lifetime 86400.
RCV:  X-- Server ID: 00:01:00:01:2e:59:d7:1b:00:1b:21:b5:6a:20
RCV:  Advertisement recorded.
PRC: Selecting best advertised lease.
PRC: Considering best lease.
PRC:  X-- Initial candidate 00:01:00:01:2e:59:d7:1b:00:1b:21:b5:6a:20 (s: 10105, p: 0).
XMT: Forming Request, 0 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Requested renew  +3600
XMT:  | X-- Requested rebind +5400
XMT:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
XMT:  | | | X-- Preferred lifetime +7200
XMT:  | | | X-- Max lifetime +7500
XMT:  V IA_NA appended.
XMT: Request on eth0, interval 1000ms.
RCV: Reply message on eth0 from fe80::8c4:60ba:1abe:3b64.
RCV:  X-- IA_NA 6d:00:00:01
RCV:  | X-- starts 1724330793
RCV:  | X-- t1 - renew  +0
RCV:  | X-- t2 - rebind +0
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
RCV:  | | | X-- Preferred lifetime 7200.
RCV:  | | | X-- Max lifetime 86400.
RCV:  X-- Server ID: 00:01:00:01:2e:59:d7:1b:00:1b:21:b5:6a:20
PRC: Bound to lease 00:01:00:01:2e:59:d7:1b:00:1b:21:b5:6a:20.
[root@rhel9-harlequin-kite-11 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 02:d8:6d:00:00:01 brd ff:ff:ff:ff:ff:ff
    altname enp1s0
    inet 10.32.111.4/20 brd 10.32.111.255 scope global dynamic noprefixroute eth0
       valid_lft 6638sec preferred_lft 6638sec
    inet6 2620:52:0:2060:dead:beef:1:6db2/128 scope global dynamic
       valid_lft 86449sec preferred_lft 7189sec
    inet6 fe80::d8:6dff:fe00:1/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
[root@rhel9-harlequin-kite-11 ~]#

rbo · 2024-08-26T17:16:49Z

Summary of some test with @dmoessne:

Change inf1 eno1 network wire from rh switch to arista-rj45 => ❌ 🫨- WTF problem still exist.
Some testing with inf31
- inf31 - eno1 - 3c:4a:92:b2:ee:08 - RH Switch ❌ bad udp cksum
- inf31 - eno2 - 3c:4a:92:b2:ee:09 - arista rj45 port11 ✅ udp sum ok
- Switch wire eno1 <-> eno2
- inf31 - eno1 - 3c:4a:92:b2:ee:08 - arista rj45 ✅
- inf31 - eno2 - 3c:4a:92:b2:ee:09 - RH Switch ❌
- Change inf31 - eno1 - 3c:4a:92:b2:ee:08 - back to rh switch => ❌
- Change inf31 - eno2 - 3c:4a:92:b2:ee:09 - with new wire to arista-rj45 => ✅
Change inf1 - eno1 - 2c:76:8a:4f:61:a4 from arista-rj45 to arista-rj45 via temp wire which works (testet at inf31) => ❌

rbo · 2024-08-26T17:24:56Z

Current configuration:

inf1 - eno1 - 2c:76:8a:4f:61:a4 arista-rj45 port Et25 : ❌
inf31 - eno1 - 3c:4a:92:b2:ee:08 - arista rj45 port Et6 : ✅

Compare ethtool -k and kernel are the same!

[root@inf1 ~]# ethtool -k eno1 > /tmp/inf1-eno.txt
[root@inf1 ~]# md5sum /tmp/inf1-eno.txt
6ff13cc56bbe76fbb2c43a5a554294ef  /tmp/inf1-eno.txt
[root@inf1 ~]# uname -a
Linux inf1.coe.muc.redhat.com 4.18.0-553.16.1.el8_10.x86_64 #1 SMP Thu Aug 1 04:16:12 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux

[root@inf31 ~]# ethtool -k eno1 > /tmp/inf31-eno.txt
[root@inf31 ~]# md5sum /tmp/inf31-eno.txt
6ff13cc56bbe76fbb2c43a5a554294ef  /tmp/inf31-eno.txt
[root@inf31 ~]# uname -a
Linux inf31.coe.muc.redhat.com 4.18.0-553.16.1.el8_10.x86_64 #1 SMP Thu Aug 1 04:16:12 EDT 2024 x86_64 x86_64 x86_64 GNU/Linux

rbo · 2024-08-26T19:11:49Z

After a reboot of inf1, I still see some udp packages with bad checksum. But dhcpd6 works!

rbo · 2024-08-26T19:13:08Z

[root@rhel9-harlequin-kite-11 ~]# rm /var/lib/dhclient/dhclient6.leases
[root@rhel9-harlequin-kite-11 ~]# dhclient -6 -v eth0
Internet Systems Consortium DHCP Client 4.4.2b1
Copyright 2004-2019 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp/

Listening on Socket/eth0
Sending on   Socket/eth0
Created duid "\000\004N'2\304Y\323D\356\2250\306\270D}n\337".
PRC: Soliciting for leases (INIT).
XMT: Forming Solicit, 0 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Request renew in  +3600
XMT:  | X-- Request rebind in +5400
XMT: Solicit on eth0, interval 1010ms.
RCV: Advertise message on eth0 from fe80::2e76:8aff:fe4f:61a4.
RCV:  X-- IA_NA 6d:00:00:01
RCV:  | X-- starts 1724699549
RCV:  | X-- t1 - renew  +0
RCV:  | X-- t2 - rebind +0
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
RCV:  | | | X-- Preferred lifetime 27000.
RCV:  | | | X-- Max lifetime 43200.
RCV:  X-- Server ID: 00:01:00:01:2e:47:d9:d2:2c:76:8a:4f:61:a4
RCV:  Advertisement recorded.
PRC: Selecting best advertised lease.
PRC: Considering best lease.
PRC:  X-- Initial candidate 00:01:00:01:2e:47:d9:d2:2c:76:8a:4f:61:a4 (s: 10105, p: 0).
XMT: Forming Request, 0 ms elapsed.
XMT:  X-- IA_NA 6d:00:00:01
XMT:  | X-- Requested renew  +3600
XMT:  | X-- Requested rebind +5400
XMT:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
XMT:  | | | X-- Preferred lifetime +7200
XMT:  | | | X-- Max lifetime +7500
XMT:  V IA_NA appended.
XMT: Request on eth0, interval 920ms.
RCV: Reply message on eth0 from fe80::2e76:8aff:fe4f:61a4.
RCV:  X-- IA_NA 6d:00:00:01
RCV:  | X-- starts 1724699550
RCV:  | X-- t1 - renew  +0
RCV:  | X-- t2 - rebind +0
RCV:  | X-- [Options]
RCV:  | | X-- IAADDR 2620:52:0:2060:dead:beef:1:6db2
RCV:  | | | X-- Preferred lifetime 7200.
RCV:  | | | X-- Max lifetime 7500.
RCV:  X-- Server ID: 00:01:00:01:2e:47:d9:d2:2c:76:8a:4f:61:a4
PRC: Bound to lease 00:01:00:01:2e:47:d9:d2:2c:76:8a:4f:61:a4.

rbo · 2024-08-26T19:48:28Z

Finally, IPv6 is enabled & configured.

COE Slack is informed.

We will continue to work on the UDP / Network cable problem.

rbo mentioned this issue Aug 2, 2024

Update ISAR to 4.16 #193

Closed

1 task

rbo self-assigned this Aug 7, 2024

rbo added enhancement New feature or request infrastructure/coe labels Aug 8, 2024

rbo mentioned this issue Aug 15, 2024

ISAR is not responsive #195

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable IPv6 #188

Enable IPv6 #188

rbo commented Jul 2, 2024 •

edited

Loading

rbo commented Aug 7, 2024

rbo commented Aug 7, 2024 •

edited

Loading

rbo commented Aug 8, 2024

rbo commented Aug 8, 2024

rbo commented Aug 9, 2024

rbo commented Aug 16, 2024

rbo commented Aug 16, 2024

rbo commented Aug 21, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 26, 2024 •

edited

Loading

rbo commented Aug 26, 2024 •

edited

Loading

rbo commented Aug 26, 2024

rbo commented Aug 26, 2024

rbo commented Aug 26, 2024

Enable IPv6 #188

Enable IPv6 #188

Comments

rbo commented Jul 2, 2024 • edited Loading

Tasks

rbo commented Aug 7, 2024

rbo commented Aug 7, 2024 • edited Loading

rbo commented Aug 8, 2024

rbo commented Aug 8, 2024

rbo commented Aug 9, 2024

rbo commented Aug 16, 2024

rbo commented Aug 16, 2024

rbo commented Aug 21, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 22, 2024

rbo commented Aug 26, 2024 • edited Loading

rbo commented Aug 26, 2024 • edited Loading

rbo commented Aug 26, 2024

rbo commented Aug 26, 2024

rbo commented Aug 26, 2024

rbo commented Jul 2, 2024 •

edited

Loading

rbo commented Aug 7, 2024 •

edited

Loading

rbo commented Aug 26, 2024 •

edited

Loading

rbo commented Aug 26, 2024 •

edited

Loading