Skip to content

Latest commit

 

History

History
424 lines (343 loc) · 15.5 KB

lab4_IPsec.adoc

File metadata and controls

424 lines (343 loc) · 15.5 KB
Table of Contents

Lab 4: Cloud Encryption Using RHEL Opportunistic IPsec

Lab Length

  • Medium/Average (~15 mins)

Goal

  • Learn how to deploy Opportunistic IPsec to encrypt all host-to-host communication within an enterprise network

Objectives

  • Generate X.509 certificates using a Python helper script

  • Configure Opportunistic IPsec on two or more nodes

  • Debug and monitor the network for IPsec encryption

4.1: Introduction

Whenever two hosts in the network send any kind of traffic, IPsec security automatically activates and encrypts all traffic between those hosts. This is also called mesh encryption.

Imagine a LAN of various machines. Each machine is given an X.509 certificate in the form of a PKCS#12 file (for example, ipsecXXX.example.com). Machines in the network automatically initiate IPsec encryption and authenticate each other based on these certificates whenever any traffic is attempted between these nodes. This is called Opportunistic IPsec.

There are two machines available to you for this lab. But because this configuration method’s main feature is scalability, you can use another X.509 certificate and repeat the process for any other provided lab VM. But be careful—​if you misconfigure IPsec and do not allow fallback to cleartext, you can lock yourself out of a VM.

4.2: Creating and Installing X.509 Certificates

In this section, you generate the required X.509 certificates. To make this easier, a script (/root/vpn-cert-gen.py) provided on ipsec.example.com generates x.509 certificates for ipsec.example.com and ipsec2.example.com for the lab. If you want to try to add more machines to the encrypted mesh network, it also creates certificates for audit.example.com and usbguard.example.com. All your actions are run as root.

  1. If you are not already there, log in to the workstation bastion host as lab-user from your desktop system (replacing GUID with your lab-provided GUID and using r3dh4t1! as the password):

    [localhost ~]$ ssh [email protected]

  2. Log in to the ipsec.example.com host as root:

    [lab-user@workstation-GUID ~]$ ssh [email protected]

  3. Create the X.509 certificates for the lab:

    [root@ipsec ~]# mkdir /root/labcerts

[root@ipsec ~]# cd /root/labcerts

[root@ipsec labcerts]# /root/vpn-cert-gen.py --wipe
creating CA cert
OK: CA system completely re-initialized

[root@ipsec labcerts]# ls
cacerts  certs  keys  mobileconfig  pkcs12  serial.txt

[root@ipsec labcerts]# ls pkcs12/
audit.example.com.p12  ipsec2.example.com.p12  ipsec.example.com.p12  servera.example.com.p12  usbguard.example.com.p12

    The cacerts/ directory contains the CA certificate.

    The certs/ directory contains the regular node certificates.

    The keys/ directory contains the private keys, but the script has already combined these for you into PKCS#12-based certificates in the pkcs12/ directory.

    The mobileconfig/ directory has Apple device profiles that include the PKCS#12 certificate and all of the other configuration items that are needed for those devices. These files are only useful when building a Remote Access VPN server. You do not use them for this mesh host-to-host encryption using Opportunistic IPsec.

  4. Select the proper pkcs12/*.p12 files for each of the hosts you want to add to the crypto mesh.

Important

Due to security issues, the certificates must have a SubjectAltName specifying the IP address they are using, to prevent one machine from impersonating another. Because my_sudo_group does not have full reverse DNS and DNSSEC in this lab, libreswan cannot link the IP address to the host name securely. This is why the Python script hard-codes the IP address in the SubjectAltName of the X.509 certificate.

4.3: Configuring Opportunistic IPsec

4.3.1: Installing and Initializing libreswan and Importing the Certificate

  1. Install libreswan and initialize the certificate database using the following commands to first set up ipsec.example.com:

    [root@ipsec ~]# yum install libreswan
[root@ipsec ~]# ipsec initnss

  2. After libreswan is installed, import the PKCS#12 certificate:

    [root@ipsec ~]# ipsec import /root/labcerts/pkcs12/ipsec.example.com.p12
Enter password for PKCS12 file:
pk12util: PKCS12 IMPORT SUCCESSFUL
correcting trust bits for Certificate Agency (CA) - Test Org

  3. When prompted for the import password, type secret.

  4. Confirm that the node’s certificate and CA certificate are installed and available to libreswan:

    [root@ipsec ~]# certutil -L -d sql:/etc/ipsec.d

Certificate Nickname                                         Trust Attributes
                                                             SSL,S/MIME,JAR/XPI

mycert                                                       u,u,u
Certificate Agency (CA) - Test Org                           CT,,

4.3.2: Configuring Hosts for IPsec

In this section, you configure the ipsec.example.com and ipsec2.example.com hosts for IPsec. For these two hosts, the configuration file is already created for you and placed on the machine at /root/oe-cert.conf.

  1. Copy this file into the /etc/ipsec.d/ directory so that libreswan can use it:

    [root@ipsec ~]# cp /root/oe-cert.conf /etc/ipsec.d/
    Note

    If you look at the /root/oe-cert.conf file, you can see it defines a number of connections. These connections are the groups that you can assign to network IP ranges. A connection specification is defined within a conn section in the configuration file.

    • conn private means that IPsec is mandatory and all plaintext is to be dropped.

    • conn private-or-clear means that IPsec is attempted, but it falls back to cleartext if it fails.

    • conn clear-or-private means that it does not initiate IPsec, but responds to a request for IPsec.

    • conn clear never allows or initiates IPsec.

  2. If you are running with SElinux enabled, make sure that all of the files are properly labeled:

    [root@ipsec ~]# restorecon -Rv /etc/ipsec.*

  3. Add an IP address (for example, 192.168.0.66) or network range (for example, 192.168.0.0/24) into one of these groups, by simply adding one line with the IP address or network (in CIDR notation) into one of the files matching the connection name in /etc/ipsec.d/policies.

  4. Configure the machines to attempt Opportunistic IPsec for the entire 192.168.0.0/24 range by adding that network range to the private-or-clear group:

    [root@ipsec ~]# echo "192.168.0.0/24" >> /etc/ipsec.d/policies/private-or-clear

  5. Add a more specific entry into the "clear" group so that workstation communication always takes place unencrypted and you can always use it to log in to other machines to reconfigure or debug:

    [root@ipsec ~]# echo "192.168.0.3/32" >> /etc/ipsec.d/policies/clear

    This allows you to log in to all of the machines via the workstation.

  6. Tell the firewalld system service that you want to open the firewall for the required packets for IPsec:

    [root@ipsec ~]# firewall-cmd --add-service=ipsec --permanent

[root@ipsec ~]# firewall-cmd --reload

4.4: Transferring the Configuration to a Second System

In this section, you configure the next machine, ipsec2.example.com. Because the ipsec.example.com host contains all of the certificates, you must copy the certificate onto ipsec2.example.com via the workstation VM:

  1. If you are not already there, log in to the workstation bastion host as lab-user from your desktop system (replacing GUID with your lab-provided GUID and using r3dh4t1! as the password):

    [localhost ~]$ ssh [email protected]

[lab-user@workstation-GUID ~]$ scp [email protected]:/root/labcerts/pkcs12/ipsec2.example.com.p12 .

[lab-user@workstation-GUID ~]$ scp [email protected]:/root/oe-cert.conf .

[lab-user@workstation-GUID ~]$ scp ipsec2.example.com.p12 oe-cert.conf [email protected]:/root/

  2. Install libreswan, import the certificate on ipsec2.example.com, and configure it for Opportunistic IPsec:

  3. Log in to the ipsec2.example.com host as root:

    [lab-user@workstation-GUID ~]$ ssh [email protected]
    [root@ipsec2 ~]# yum install libreswan

[root@ipsec2 ~]# ipsec initnss

[root@ipsec2 ~]# ipsec import /root/ipsec2.example.com.p12
(Note: The password for PKCS12 file is "secret".)

[root@ipsec2 ~]# rm /root/ipsec2.example.com.p12

[root@ipsec2 ~]# mv /root/oe-cert.conf /etc/ipsec.d/

[root@ipsec2 ~]# restorecon -Rv /etc/ipsec.d

[root@ipsec2 ~]# echo "192.168.0.0/24" >> /etc/ipsec.d/policies/private-or-clear

[root@ipsec2 ~]# echo "192.168.0.3/32" >> /etc/ipsec.d/policies/clear

[root@ipsec2 ~]# firewall-cmd --add-service=ipsec --permanent

[root@ipsec2 ~]# firewall-cmd --reload

    Now you have configured the first two nodes. For each additional node, all you need to do is generate and install a new certificate, add the same configuration file with the updated leftcert= entry, and update the policy groups in /etc/ipsec.d/policies/ to match the first two nodes of the cluster. So for each added node, you do not need to reconfigure any of the previous nodes, as those are already configured to trust the same CA and talk IPsec to the same IP ranges as the new nodes. Because all of the configuration files (other then the PKCS#12 certificate) are identical on all or the nodes, in a production setting, you should be able to populate new nodes using Ansible®.

4.5: Using Opportunistic IPsec

You are now ready to test the configuration.

First, you start the IPsec subsystem on both of the hosts.

  1. Start the IPsec subsystem on ipsec.example.com:

    [root@ipsec ~]# systemctl start ipsec

  2. Start the IPsec subsystem on ipsec2.example.com:

    [root@ipsec2 ~]# systemctl start ipsec

  3. Wait a few seconds for the two hosts to load their IPsec policies.

  4. On ipsec.example.com, ping ipsec2.example.com from ipsec.example.com to trigger an IPsec tunnel:

    [root@ipsec ~]# ping -c3 ipsec2.example.com

    The first ping may fail depending on the time it takes to set up the IPsec connection.

  5. Examine the system logs in /var/log/secure, or you can use one of the various status commands available:

    [root@ipsec ~]# ipsec trafficstatus
006 #2: "private-or-clear#192.168.0.0/24"[1] ...192.168.0.22, type=ESP, add_time=1523268130, inBytes=1848, outBytes=1848, id='C=CA, ST=Ontario, L=Toronto, O=Test Org, OU=Clients, CN=ipsec.example.com, [email protected]'

  6. Examine the output to see the non-zero byte counters for IPsec packets that shows that the kernel IPsec subsystem has encrypted and decrypted network packets.

    Tip

    A more verbose command is:

    [root@ipsec ~]# ipsec status
<lots of output>

Your two nodes now have an IPsec-encrypted mesh network running.

4.6: Troubleshooting (Optional)

  1. If you think something went wrong and the ipsec status command does not show you the private, private-or-clear, and clear-or-private connections (and their instances), issue a manual command to see why loading failed:

    [root@ipsec ~]# ipsec auto --add private

  2. If there is some kind of failure (such as the group is private but the remote end is not functioning), do not expect to see a visible IPsec tunnel, but examine the results and expect to see the shunts that prevent or allow unencrypted traffic on the network:

    [root@ipsec ~]# ipsec whack --shuntstatus
000 Bare Shunt list:
000
000 192.168.0.23/32:0 -0-> 192.168.0.22/32:0 => %drop 0    oe-failing
    Note

    There are a few kinds of shunts. The negotiationshunt determines what to do with packets while the IPsec connection is being established. Usually people want to hold the packets to prevent leaks, but if encryption is only "nice to have" and an uninterrupted service is more important, you can set this option to passthrough. The failureshunt option determines what to do when negotiation fails. For the private-or-clear entry in your configuration file, you can see it is set to passthrough, allowing unencrypted traffic. For the private entry you can see it is set to drop to disallow unencrypted traffic.

  3. Run ping on one host and tcpdump on the other host to confirm that the connection is encrypted:

    [root@ipsec ~]# tcpdump -i ens3 -n esp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes
05:58:18.003410 IP 192.168.0.22 > 192.168.0.23: ESP(spi=0x84019944,seq=0x6), length 120
05:58:18.003684 IP 192.168.0.23 > 192.168.0.22: ESP(spi=0x5b312cc5,seq=0x6), length 120
05:58:19.004840 IP 192.168.0.22 > 192.168.0.23: ESP(spi=0x84019944,seq=0x7), length 120
05:58:19.005096 IP 192.168.0.23 > 192.168.0.22: ESP(spi=0x5b312cc5,seq=0x7), length 120
05:58:20.006529 IP 192.168.0.22 > 192.168.0.23: ESP(spi=0x84019944,seq=0x8), length 120
05:58:20.006730 IP 192.168.0.23 > 192.168.0.22: ESP(spi=0x5b312cc5,seq=0x8), length 120

    If you see ESP packets with tcpdump, it means the connection is sending encrypted traffic. If you use ping and see ICMP packets, then the connection is not encrypted. Due to how the kernel hooks for IPsec and tcpdump interact, if you look at all traffic over an interface, you might see unencrypted packets going out and encrypted (proto ESP) and decrypted packets coming in. This happens because packets are encrypted by IPsec after the tcpdump hook has seen the packet on some kernel versions. The easiest indicator of whether traffic is encrypted is to use the above-mentioned trafficstatus command.

  4. Simply repeat this process on any new node to create your crypto mesh.

    If you added the entire network range (192.168.0.0/24) to the private or private-or-clear groups, then for every new node you add, you do not need to reconfigure anything on the existing node.

  5. Optionally, redo the test—​but this time, do not run libreswan on one node—​then perform a ping.

    Expect to see a few packets stalled or failing (based on whether the IP or subnet appears in /etc/ipsec.d/policies/private or /etc/ipsec.d/policies/private-or-clear) before it fails to clear or installs a block.

  6. Additionally, if you run into more problems or you want to see in detail what is happening, enable two lines in /etc/ipsec.conf to get all of the logs in a file with full debugging:

    # example /etc/ipsec.conf
config setup
 logfile=/var/log/pluto.log
 plutodebug=all

include /etc/ipsec.d/*.conf

    It is important to use file logging with full debugging because otherwise the rsyslog or systemd ratelimit may cause you to miss messages.

  7. If everything works as expected, enable the IPsec services on your cluster on every startup—​on each node run:

    [root@ipsec ~]# systemctl enable ipsec

For more information on Opportunistic IPsec, see Opportunistic IPsec.

4.7: Resetting the IPsec NSS Certificate Database (Optional)

Libreswan uses the NDD cryptographic library. It keeps all of its X.509 certificates and keys in its own NSS database in /etc/ipsec.d.

  1. If you want restart the entire lab from scratch, run the following commands to remove the entire libreswan NSS database:

    Warning

    Executing these commands requires you to restart the IPsec lab from the beginning.

    		 
    [root@ipsec ~]# systemctl stop ipsec
[root@ipsec ~]# rm /etc/ipsec.d/*.db
[root@ipsec ~]# ipsec initnss
Initializing NSS database

  2. Follow the same procedure for ipsec2.example.com.