User-Name in RADIUS message is arbitrary

[G-Venkata-Surya-Sai]

Motive:
At our campus data-center we are implementing zero-trust network architecture, where each of the instances, like bare-metal servers, VMs/containers running on hypervisors (QEMU-KVM/LXD), have to go through network access control (802.1x) to get Internet connectivity. For bare-metal servers, we have Altoline Whitebox switches that can run Open Virtual Switch (OVS). For VMs and Containers, OVS will be running inside hypervisors. We are experimenting with Faucet SDN, which already has the experimental 802.1x feature (Chewie).

Description:
while verifying 802.1x port-based authentication using faucet/chewie, we notice arbitrary mac address in the username field of RADIUS messages to the radius server.

Prototype setup:
This prototype setup is inside an All-In-One VM named “faucet-chewie-aio” with Debian 11 operating system. Inside this VM, we are running 3 LXD containers named cont0, cont1, and cont2, all of them running ubuntu 22.04 LTS server (headless systems).

[G-Venkata-Surya-Sai]

AIO VM playing the role of the hypervisor.

Network settings in the VM:

root@faucet-chewie-aio:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:20:aa:0d brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 192.168.51.79/23 brd 192.168.51.255 scope global ens3
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe20:aa0d/64 scope link
       valid_lft forever preferred_lft forever
3: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:dc:b8:79 brd ff:ff:ff:ff:ff:ff
    altname enp0s4
    inet 192.168.47.79/23 brd 192.168.47.255 scope global ens4
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fedc:b879/64 scope link
       valid_lft forever preferred_lft forever
4: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master ovs-system state UP group default qlen 1000
    link/ether 52:54:00:c8:20:59 brd ff:ff:ff:ff:ff:ff
    altname enp0s5
    inet 10.200.0.79/22 brd 10.200.3.255 scope global ens5
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fec8:2059/64 scope link
       valid_lft forever preferred_lft forever
5: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 76:39:8b:6d:e0:cd brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 72:93:ba:45:a2:42 brd ff:ff:ff:ff:ff:ff
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:5d:30:ee:76 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
8: br-2d6c67783f7d: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:b7:fa:d9:7f brd ff:ff:ff:ff:ff:ff
    inet 172.24.0.1/16 brd 172.24.255.255 scope global br-2d6c67783f7d
       valid_lft forever preferred_lft forever
9: dm-857f893be141: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    link/ether 0a:1b:45:a9:90:82 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b88a:faff:fe64:ec6b/64 scope link
       valid_lft forever preferred_lft forever
43: vethd335b8dc@if42: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether e2:dc:88:d7:05:f3 brd ff:ff:ff:ff:ff:ff link-netnsid 0
45: vethac0ae1e2@if44: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 86:48:33:0e:a6:8d brd ff:ff:ff:ff:ff:ff link-netnsid 1
47: veth551adc11@if46: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master ovs-system state UP group default qlen 1000
    link/ether 12:07:d1:7c:20:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 2

root@faucet-chewie-aio:~# ip r
default via 192.168.50.1 dev ens3 proto static
10.200.0.0/22 dev ens5 proto kernel scope link src 10.200.0.79
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
172.24.0.0/16 dev br-2d6c67783f7d proto kernel scope link src 172.24.0.1 linkdown
192.168.46.0/23 dev ens4 proto kernel scope link src 192.168.47.79
192.168.50.0/23 dev ens3 proto kernel scope link src 192.168.51.79

The three containers are

root@faucet-chewie-aio:~# lxc ls
+-------+---------+--------------------+------+-----------+-----------+
| NAME  |  STATE  |        IPV4        | IPV6 |   TYPE    | SNAPSHOTS |
+-------+---------+--------------------+------+-----------+-----------+
| cont0 | RUNNING |                    |      | CONTAINER | 0         |
+-------+---------+--------------------+------+-----------+-----------+
| cont1 | RUNNING | 10.200.3.226 (cn0) |      | CONTAINER | 0         |
+-------+---------+--------------------+------+-----------+-----------+
| cont2 | RUNNING | 10.200.3.227 (cn0) |      | CONTAINER | 0         |
+-------+---------+--------------------+------+-----------+-----------+

These containers are connected to ovs-bridge named br0.

root@faucet-chewie-aio:~# ovs-vsctl show
be4fcfbf-7f49-477f-9159-1f2517ad78cb
    Bridge br0
        Controller "tcp:127.0.0.1:6654"
            is_connected: true
        Controller "tcp:127.0.0.1:6653"
            is_connected: true
        fail_mode: secure
        Port ens5
            Interface ens5
        Port vethd335b8dc
            Interface vethd335b8dc
        Port br0
            Interface br0
                type: internal
        Port vethac0ae1e2
            Interface vethac0ae1e2
        Port veth551adc11
            Interface veth551adc11
    ovs_version: "2.15.0"


root@faucet-chewie-aio:~# ovs-dpctl show
system@ovs-system:
  lookups: hit:1851490 missed:10026 lost:0
  flows: 3
  masks: hit:1869147 total:2 hit/pkt:1.00
  port 0: ovs-system (internal)
  port 1: br0 (internal)
  port 2: vethd335b8dc
  port 3: vethac0ae1e2
  port 4: veth551adc11
  port 5: ens5

Faucet is running locally in the AIO VM.

root@faucet-chewie-aio:~# systemctl status faucet
● faucet.service - "Faucet OpenFlow switch controller"
     Loaded: loaded (/lib/systemd/system/faucet.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-12-07 08:25:39 IST; 4h 18min ago
       Docs: https://docs.faucet.nz
   Main PID: 63796 (osken-manager)
      Tasks: 1 (limit: 9508)
     Memory: 68.0M
        CPU: 25.751s
     CGroup: /system.slice/faucet.service
             └─63796 /usr/bin/python3 /usr/bin/osken-manager --config-file=/etc/faucet/ryu.conf --ofp-tcp-listen-port=6653 faucet.faucet


root@faucet-chewie-aio:~# netstat -tunlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:6653            0.0.0.0:*               LISTEN      63796/python3       
tcp        0      0 0.0.0.0:6654            0.0.0.0:*               LISTEN      1161/python3        
tcp        0      0 127.0.0.1:41903         0.0.0.0:*               LISTEN      685/containerd      
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/init              
tcp        0      0 0.0.0.0:9302            0.0.0.0:*               LISTEN      63796/python3       
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      686/sshd: /usr/sbin
tcp        0      0 0.0.0.0:9303            0.0.0.0:*               LISTEN      1161/python3        
tcp6       0      0 :::9090                 :::*                    LISTEN      501/prometheus      
tcp6       0      0 :::9100                 :::*                    LISTEN      498/prometheus-node
tcp6       0      0 :::111                  :::*                    LISTEN      1/init              
tcp6       0      0 :::22                   :::*                    LISTEN      686/sshd: /usr/sbin
tcp6       0      0 :::3000                 :::*                    LISTEN      1162/grafana-server
udp        0      0 0.0.0.0:51948           0.0.0.0:*                           63796/python3       
udp        0      0 0.0.0.0:111             0.0.0.0:*                           1/init              
udp6       0      0 :::111                  :::*                                1/init              

Authentication server:

Radius server is present outside the VM. with IP 192.168.50.7.
Radius server is working fine. I tested this using radtest with the above user id(user1) and password(topsecret).

root@faucet-chewie-aio:~# radtest -x user1 topsecret 192.168.50.7 -0 testing123 radiussrv
Sent Access-Request Id 142 from 0.0.0.0:55817 to 192.168.50.7:1812 length 84
        User-Name = "user1"
        User-Password = "topsecret"
        NAS-IP-Address = 127.0.1.1
        NAS-Port = 0
        Message-Authenticator = 0x00
        Framed-Protocol = PPP
        Cleartext-Password = "topsecret"
Received Access-Accept Id 142 from 192.168.50.7:1812 to 192.168.51.79:55817 length 32
        Framed-Protocol = PPP
        Framed-Compression = Van-Jacobson-TCP-IP

Supplicant:

Cont0 is the EAPOL supplicant, generating EAPOL packets from the netplan file.

root@cont0:~# vi /etc/netplan/10-lxc.yaml 
network:
  version: 2
  ethernets:
    cn0:
      auth:
        key-management: 802.1x
        method: ttls
        identity: "user1"
        password: "topsecret”
      dhcp4: true

Authenticator:

Faucet/chewie act as authenticator. Faucet is running locally inside the VM at 127.0.0.1.

root@faucet-chewie-aio:~# vi /etc/faucet/faucet.yaml
---
vlans:
    office:
        vid: 560
        description: "overlay host network"
dps:
    sw1:
        dot1x:
            radius_ip: "192.168.50.7"
            radius_port: 1812
            radius_secret: "testing123"
            nfv_intf: "ens3"
            nfv_sw_port: 1
        dp_id: 0x1
        hardware: "Open vSwitch"
        interfaces:
            1:
                name: "DMZ"
                description: "DMZ"
                output_only: true
            4:
                name: "ens5"
                description: "trunk"
                native_vlan: office
            18:
                name: "cont0"
                description: "cont0"
                native_vlan: office
                dot1x: true
            19:
                name: "cont1"
                description: "cont1"
                native_vlan: office
            20:
                name: "cont2"
                description: "cont2"
                native_vlan: office
                     

I got these port numbers 4,18,19,20 from the “/var/log/faucet/faucet.log” file. Before adding dot1x, all these containers are getting DHCP IP. So we can confirm that these ports are correct. I arbitrarily assigned nfv_sw_port to 1.

[G-Venkata-Surya-Sai]

Results:

Collected packet traces in port 18, ens3, ens5, lo

On Port 18(Port connected to cont0):
I could notice EAPOL-start, but there is no further handshake.

On ens3 Port:
We could see RADIUS server fails with Access-Reject. It failed because the username value is not the same as what we are sending.
Username inside RADIUS packet is some arbitrary mac address “008e732690c7” instead of “user1”.

On port ens5, there are no EAPOL and radius packets.
On port lo, there are openflow messages. But there are no EAPOL, RADIUS messages.

Issues:

1. In the EAP frame, why is User-Name some arbitrary mac-address instead of “identity” given in the netplan file? If I use the same netplan file inside a bare-metal server which is connected to bare-metal switch (cisco sg350) with 802.1x enabled as the authenticator, then eapol-radius hand-shake goes through, and DHCP server allots IP within a jiffy. What could be the cause for this arbitrary mac address in user-name?
2. What number should I write for nfv_sw_port in the faucet.yaml?
3. Port numbers used by faucet (which are reported in the /var/log/faucet/faucet log) are changing after each hypervisor-reboot. This way we need to change faucet.yaml file every time after reboot. This is not feasible at the production level. Is there any solution for port persistence?

[BalG10]

I found a similar issue to yours as well but in my case Faucet also terminates. Below is the error message as well as faucet terminating due to the failed authentication between radius and chewie