initial commit of new workshop

.gitignore

@@ -0,0 +1 @@
+.vagrant

README.md

@@ -1,14 +1,14 @@
-# rear-workshop-osbconf-2016
+# Relax-and-Recover (ReaR) Workshop
 
 :author: Gratien Dhaese <[email protected]>
 
-The Relax-and-Recover (rear) workshop for OSBconf 2016 (Open Source Backup Conference, Cologne, Germany, 26-27 September, 2016) - see http://osbconf.org/workshops/ - is guiding you into setting up rear, how to configure it, and with lots of real use cases, such as with Bareos, NFS, CIFS, RSYNC.
+The Relax-and-Recover (ReaR) workshop is guiding you into setting up rear, how to configure it, and with lots of real use cases, such as with Bareos, NFS, CIFS, RSYNC.
 
 ## Setting up your virtual machines
 
 This document was only tested for the KVM and VirtualBox hypervisors, but it should work fine with other hypervisors.
 
-I would really appreciate that you test your hypervisor of choice and contribute instructions back (at https://github.com/rear/rear-workshop-osbconf-2016/pulls).
+I would really appreciate that you test your hypervisor of choice and contribute instructions back (at https://github.com/rear/rear-workshop/pulls).
 
 A few words before going to the prerequisites - we use **vagrant** as digital assitant to setup up our virtual images (VMs) to play with rear. Why? Because, in order to speed up the workshop around rear we do install lots of software automatically in the client and server VMs.
 
@@ -21,10 +21,11 @@ To give an example, when you have setup the _client_ and _server_ VMs by running
 Before we can start with the content of the workshop you need several things:
 
  - Host system can be Linux, Mac, Windows
- - A hypervisor like KVM, VirtualBox, VMware Player or VMware Fusion, Parallels Desktop
+ - A hypervisor like KVM, Oracle VirtualBox, VMware Player or VMware Fusion, Parallels Desktop
  - Install *vagrant* from your distribution or when not present from https://www.vagrantup.com/downloads.html
  - KVM with libvirt needs the *vagrant-libvirt* plugin:  _vagrant plugin install vagrant-libvirt_
- - Install *git* to download the workshop: _git clone https://github.com/rear/rear-workshop-osbconf-2016.git_
+ - Install *git* to download the workshop: _git clone https://github.com/rear/rear-workshop.git_
+   or _git clone [email protected]:rear/rear-workshop.git_
  - Sufficient free disk space for 3 VMs (about 3G per virtual machine should do)
  - Optional, vncviewer to approach the recover VM
 
@@ -36,8 +37,8 @@ At this point we assume you have a hypervisor and vagrant already installed. Als
 Start with downloading the workshop:
 
 <pre>
-$ git clone https://github.com/rear/rear-workshop-osbconf-2016.git
-Cloning into 'rear-workshop-osbconf-2016'...
+$ git clone https://github.com/rear/rear-workshop.git
+Cloning into 'rear-workshop'...
 remote: Counting objects: 160, done.
 remote: Total 160 (delta 0), reused 0 (delta 0), pack-reused 160
 Receiving objects: 100% (160/160), 40.76 KiB | 0 bytes/s, done.
@@ -48,7 +49,7 @@ Checking connectivity... done.
 Then, browse into:
 
 <pre>
-$ cd rear-workshop-osbconf-2016/centos7/
+$ cd rear-workshop/centos7/
 $ ls
 nodeconfig-centos7.sh  provision-centos7.sh  Vagrantfile  Vagrantfile.libvirt.recover  Vagrantfile.virtualbox.recover
 </pre>
@@ -102,7 +103,7 @@ There are several possibilities to login onto these fresh created VMs:
  - vncviewer 127.0.0.1:5991   (for the client interface)
  - vncviewer 127.0.0.1:5992   (for the server interface)
  - vncviewer 127.0.0.1:5993   (for the recover interface)
- - Or, via the console of your hypervisor
+ - Or, via the VM console of your hypervisor
 
 The passwords for the _vagrant_ and _root_ user are the same: *vagrant*
 
@@ -123,7 +124,7 @@ And, the vagrant box can be removed as *vagrant box remove centos/7*
 
 # Encountered a problem with setting up the workshop
 
-Oops, please open a new issue at https://github.com/rear/rear-workshop-osbconf-2016/issues
+Oops, please open a new issue at https://github.com/rear/rear-workshop/issues
 
 # Known Issues
 
@@ -132,13 +133,13 @@ Oops, please open a new issue at https://github.com/rear/rear-workshop-osbconf-2
 When you get to see an error like the following:
 
 <pre>
-=> client: Rsyncing folder: /home/grati/rear-workshop-osbconf-2016/centos7/ => /vagrant
+=> client: Rsyncing folder: /home/grati/rear-workshop/centos7/ => /vagrant
 There was an error when attempting to rsync a synced folder.
 Please inspect the error message below for more info.
 
-Host path: /home/grati/rear-workshop-osbconf-2016/centos7/
+Host path: /home/grati/rear-workshop/centos7/
 Guest path: /vagrant
-Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/cygwin64/tmp/ssh.977 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/cygwin64/home/grati/rear-workshop-osbconf-2016/insecure_keys/vagrant.private' --exclude .vagrant/ /home/grati/rear-workshop-osbconf-2016/centos7/ [email protected]:/vagrant
+Command: rsync --verbose --archive --delete -z --copy-links --chmod=ugo=rwX --no-perms --no-owner --no-group --rsync-path sudo rsync -e ssh -p 2222 -o ControlMaster=auto -o ControlPath=C:/cygwin64/tmp/ssh.977 -o ControlPersist=10m -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i 'C:/cygwin64/home/grati/rear-workshop/insecure_keys/vagrant.private' --exclude .vagrant/ /home/grati/rear-workshop/centos7/ [email protected]:/vagrant
 Error: Warning: Permanently added '[127.0.0.1]:2222' (ECDSA) to the list of known hosts.
 mm_receive_fd: no message header
 process_mux_new_session: failed to receive fd 0 from slave
@@ -157,4 +158,4 @@ If you need to contact me for setting a workshop on your premises then see the p
 
 Be aware, this workshop uses *centos/7* as GNU/Linux Operating system. If you want to have it for another version or type of GNU/Linux then you have to pay for it (consultancy fee - see above link).
 
-Last updated: 09 September 2016
+Last updated: 30 December 2016

centos7/Vagrantfile

@@ -2,8 +2,8 @@
 # vi: set ft=ruby :
 
 nodes = [
-   { :hostname => 'client', :ip => '192.168.33.10', :box => 'centos/7', :ram => 1024, :gport => '5991' },
-   { :hostname => 'server', :ip => '192.168.33.15', :box => 'centos/7', :ram => 1024, :gport => '5992' }
+   { :hostname => 'client', :ip => '192.168.33.10', :box => 'centos/7', :ram => 1024 },
+   { :hostname => 'server', :ip => '192.168.33.15', :box => 'centos/7', :ram => 1024 }
 ]
 
 Vagrant.configure(2) do |config|
@@ -13,19 +13,13 @@ Vagrant.configure(2) do |config|
 
   nodes.each do |node|
     config.vm.define node[:hostname] do |nodeconfig|
-      nodeconfig.vm.box = node[:box]
+      nodeconfig.vm.box = "centos/7"
       nodeconfig.vm.hostname = node[:hostname] + ".box"
       nodeconfig.vm.network :private_network, ip: node[:ip]
-      # we will stick with the default insecure ssh keys
-      nodeconfig.ssh.insert_key = false
-      nodeconfig.ssh.private_key_path = "../insecure_keys/vagrant.private"
 
       memory = node[:ram] ? node[:ram] : 256;
-
-      nodeconfig.vm.provider :libvirt do |libvirt|
-        # with vncviewer we can connect to client, server via their specific port definition
-        # we can also use vagrant ssh [client|server] of course
-        libvirt.graphics_port = node[:gport]
+      nodeconfig.vm.provider :virtualbox do |vb|
+        #
       end
 
       nodeconfig.vm.synced_folder ".", "/home/vagrant/sync", disabled: true
@@ -35,31 +29,37 @@ Vagrant.configure(2) do |config|
       nodeconfig.vm.provision :shell, path: "nodeconfig-centos7.sh"
     # end of config.vm.define node
     end
-
 
 
   # end of nodes.each do |node|
   end
 
   config.vm.define "recover", autostart: false do |recover|
+    recover.vm.box = "clink15/pxe"
+    recover.vm.boot_timeout = 3600
+    recover.ssh.insert_key = "false"
+
+    #recover.vm.network "public_network"
     recover.vm.network :private_network, ip: "192.168.33.5"
-    # recover.ssh.username = "root"
-    recover.ssh.insert_key = false
-    recover.ssh.private_key_path = "../insecure_keys/vagrant.private"
+    recover.ssh.username = "root"
 
-    recover.vm.provider :libvirt do |recover_prov|
-      recover_prov.graphics_port = 5993
-      recover_prov.graphics_ip = '127.0.0.1'
-      recover_prov.video_type = 'qxl'
-      recover_prov.storage :file, :size => '40G', :type => 'raw'
-      # make sure file "/exports/isos/client/rear-client.iso" is readable by others
-      recover_prov.storage :file, :device => :cdrom, :path => '/exports/isos/client/rear-client.iso'
-      recover_prov.boot 'cdrom'
-      recover_prov.boot 'hd'
-      recover_prov.boot 'network'
+    recover.vm.provider :virtualbox do |recover_prov|
+      recover_prov.name = "recover"
+      recover_prov.gui = true unless ENV['NO_GUI']
+      recover_prov.memory = "1024"
+      recover_prov.cpus = "1"
+
+      # https://docs.vagrantup.com/v2/virtualbox/configuration.html
+      # http://superuser.com/questions/741734/virtualbox-how-can-i-add-mount-an-iso-image-file-from-command-line
+      # make sure file "/export/isos/client/rear-client.iso" is readable by others
+      # According URL https://github.com/mitchellh/vagrant/issues/8105 "IDE Controller" might be "IDE" only
+      recover_prov.customize ["storageattach", :id, "--storagectl", "IDE",
+                              "--port", "0", "--device", "1", "--type", "dvddrive",
+                              "--medium", "/export/isos/client/rear-client.iso"]
+      recover_prov.customize ["modifyvm", :id, "--boot1", "DVD", "--boot2", "disk",
+                              "--boot3", "none", "--boot4", "none"]
       # end of |recover_prov|
       end
-
     # end of |recover| 
     end
 

centos7/Vagrantfile.virtualbox.recover

@@ -18,8 +18,9 @@ Vagrant.configure(2) do |config|
       nodeconfig.vm.network :private_network, ip: node[:ip]
 
       memory = node[:ram] ? node[:ram] : 256;
-      #nodeconfig.vm.provider :libvirt do |libvirt|
-      #end
+      nodeconfig.vm.provider :virtualbox do |vb|
+        #
+      end
 
       nodeconfig.vm.synced_folder ".", "/home/vagrant/sync", disabled: true
       nodeconfig.vm.provision :shell, path: "provision-centos7.sh"
@@ -34,21 +35,29 @@ Vagrant.configure(2) do |config|
   end
 
   config.vm.define "recover", autostart: false do |recover|
+    recover.vm.box = "clink15/pxe"
+    recover.vm.boot_timeout = 3600
+    recover.ssh.insert_key = "false"
+
+    #recover.vm.network "public_network"
     recover.vm.network :private_network, ip: "192.168.33.5"
     recover.ssh.username = "root"
 
-    recover.vm.provider :libvirt do |recover_prov|
-      recover_prov.gui = true
+    recover.vm.provider :virtualbox do |recover_prov|
+      recover_prov.name = "recover"
+      recover_prov.gui = true unless ENV['NO_GUI']
+      recover_prov.memory = "1024"
+      recover_prov.cpus = "1"
+
       # https://docs.vagrantup.com/v2/virtualbox/configuration.html
       # http://superuser.com/questions/741734/virtualbox-how-can-i-add-mount-an-iso-image-file-from-command-line
-      # make sure file "/exports/isos/client/rear-client.iso" is readable by others
-      recover_prov.customize ["storageattach", :id, "--storagectl", "IDE Controller",
+      # make sure file "/export/isos/client/rear-client.iso" is readable by others
+      # According URL https://github.com/mitchellh/vagrant/issues/8105 "IDE Controller" might be "IDE" only
+      recover_prov.customize ["storageattach", :id, "--storagectl", "IDE",
                               "--port", "0", "--device", "1", "--type", "dvddrive",
-                              "--medium", "/exports/isos/client/rear-client.iso"]
+                              "--medium", "/export/isos/client/rear-client.iso"]
       recover_prov.customize ["modifyvm", :id, "--boot1", "DVD", "--boot2", "disk",
                               "--boot3", "none", "--boot4", "none"]
-      # recover_prov.memory = 1024
-      # recover_prov.cpus = 2
       # end of |recover_prov|
       end
     # end of |recover| 

centos7/nodeconfig-centos7.sh

@@ -15,6 +15,12 @@ yum install -y bareos-filedaemon bareos-bconsole
 #yum install -y bareos-client-conf # fails see issue #1
 rpm -i --replacefiles $( ls /srv/http/packages/workshop/bareos-client-conf-*.rpm )
 
+# check if eth1 is UP
+# Why? See issue https://github.com/mitchellh/vagrant/issues/8115
+# Will probably be fixed in vagrant 1.9.2
+# Currently, when we start the vm (without provisioning) the eth1 is still DOWN
+ip addr show dev eth1 | grep -q DOWN && systemctl restart network.service
+
 echo "Enabling and Starting bareos-fd daemon process"
 systemctl enable bareos-fd.service
 systemctl start bareos-fd.service
@@ -34,6 +40,9 @@ cat > /etc/exports <<EOF
 /export/nfs 192.168.0.0/16(rw,no_root_squash)
 EOF
 
+# check if eth1 is UP
+ip addr show dev eth1 | grep -q DOWN && systemctl restart network.service
+
 # start the NFS service
 systemctl start  nfs-idmapd rpcbind nfs-server
 systemctl enable nfs-idmapd rpcbind nfs-server

centos7/provision-centos7.sh

@@ -10,14 +10,18 @@ timedatectl set-timezone Europe/Brussels
 date
 
 
-# Add IP address to /etc/hosts
+# Add IP addresses to /etc/hosts
 IPclient="192.168.33.10"
 grep -q "^$IPclient" /etc/hosts
 if [[ $? -eq 1 ]] ;then
    echo "Add IP addresses of client and server to /etc/hosts file"
+   echo "# Private IP addresses are in the 192.168.33.0 network" >> /etc/hosts
    echo "192.168.33.10   client.box client vagrant-client" >> /etc/hosts
    echo "192.168.33.15   server.box server vagrant-server" >> /etc/hosts
-   echo "192.168.33.1    vagrant-host" >> /etc/hosts
+   echo "192.168.33.1    vagrant-host vagrant-host-private" >> /etc/hosts
+   echo "# Oracle VirtualBox Public IP addresses are using network 10.0.2.0" >> /etc/hosts
+   echo "# With Oracle VirtualBox the public interface eth0 uses IP address 10.0.2.15" >> /etc/hosts
+   echo "10.0.2.2   	 vagrant-host-public" >> /etc/hosts
 fi
 
 
@@ -101,6 +105,7 @@ sed -i 's/Defaults\s*requiretty/Defaults !requiretty/' /etc/sudoers
 # SSH setup
 # Add Vagrant ssh key for root and vagrant accouts.
 sed -i 's/.*UseDNS.*/UseDNS no/' /etc/ssh/sshd_config
+sed -i 's/.*PasswordAuthentication.*/PasswordAuthentication yes/' /etc/ssh/sshd_config
 
 # setup SSH keys for user root
 [ -d ~root/.ssh ] || mkdir  -m 700 ~root/.ssh
@@ -151,12 +156,20 @@ fi
 echo "Current mode of SELinux is \"$(getenforce)\"" 
 echo "Give httpd rights to read from /srv/http/packages"
 chcon -R -t httpd_sys_content_t /srv/http/packages/
+# However, SELinux and Oracle VirtualBox are not close friends and give restore issues when enabled
+# Therefore, for the sake of the workshop we disable it
+echo "Force SELinux in Permissive mode"
+setenforce Permissive
 
 # Restart httpd so it knows about $RPMDIR location (/etc/httpd/conf.d/packages.conf)
 #service httpd restart
 /bin/systemctl restart  httpd.service
 /bin/systemctl enable   httpd.service
 
+# Disacle kdump service - fails anyway due to lack of swap
+/bin/systemctl stop kdump.service
+/bin/systemctl disbale kdump.service
+
 # add the domain name to the /etc/idmapd.conf file (for NFSv4)
 sed -i -e 's,^#Domain =.*,Domain = box,' /etc/idmapd.conf
 

rear-configs/local-with-bareos.conf

@@ -1,5 +1,4 @@
 # The rear configuration file required during the workshop
-# at OSBconf 2016 - lab bareos
 # Actions to be taken:
 # 1/ take a full backup of the client with bareos - use bconsole
 # 2/ cp /etc/rear/workshop/local-with-bareos.conf /etc/rear/local.conf
@@ -11,9 +10,13 @@
 #    connect to the recover with address 127.0.0.1:5993)
 
 OUTPUT=ISO
+# use vagrant-host-public with Oracle VirtualBox
+#OUTPUT_URL=nfs://vagrant-host-public/exports/isos
+# use vagrant-host with libvirt
 OUTPUT_URL=nfs://vagrant-host/exports/isos
 ISO_DEFAULT=manual
 BACKUP=BAREOS
 USE_STATIC_NETWORKING=y
+KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"
 #BAREOS_RESTORE_JOB=client-restore
 #BAREOS_FILESET=client-fileset

rear-configs/local-with-cifs.conf

@@ -1,5 +1,5 @@
 # The rear configuration file required during the workshop
-# at OSBconf 2016 - lab with BACKUP=NETFS
+# lab with BACKUP=NETFS
 # Actions to be taken:
 # 1/ copy /etc/rear/workshop/local-with-cifs.conf to
 #    /etc/rear/local.conf
@@ -16,3 +16,4 @@ BACKUP=NETFS
 BACKUP_URL=cifs://server/homes
 BACKUP_OPTIONS="cred=/etc/rear/.cifs"
 USE_STATIC_NETWORKING=y
+KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"

rear-configs/local-with-dir-excludes.conf

@@ -5,6 +5,6 @@
 OUTPUT=ISO
 BACKUP=NETFS
 NETFS_KEEP_OLD_BACKUP_COPY=no
-NETFS_URL=nfs://10.1.30.20/u02/rear_backups
+BACKUP_URL=nfs://10.1.30.20/u02/rear_backups
 BACKUP_PROG_EXCLUDE=( '/u02/*' '/u03/*' '/u04/*' '/u05/*' '/u06/*' '/u07/*' '/u08/*' '/u09/*' '/u10/*' '/u11/* '/u12/*' '/u13/*' '/u14/*' )
 

rear-configs/local-with-nfs.conf

@@ -1,5 +1,5 @@
 # The rear configuration file required during the workshop
-# at OSBconf 2016 - lab backup with nfs
+# lab backup with nfs
 # Actions to be taken:
 # 1/ copy this config to /etc/rear/local.conf
 # 2/ use this config file to run 'rear -v mkbackup'
@@ -13,5 +13,7 @@ OUTPUT_URL=nfs://vagrant-host/exports/isos
 # by default it is normal the first HD (boothd)
 ISO_DEFAULT=manual
 BACKUP=NETFS
-BACKUP_URL=nfs://root@server/export/nfs
+BACKUP_URL=nfs://server/export/nfs
 USE_STATIC_NETWORKING=y
+KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"
+

rear-configs/local-with-rsync.conf

@@ -1,5 +1,5 @@
 # The rear configuration file required during the workshop
-# at OSBconf 2016 - lab with BACKUP=RSYNC
+# lab with BACKUP=RSYNC
 # Actions to be taken:
 # 1/ copy /etc/rear/workshop/local-with-rsync.conf to
 #    /etc/rear/local.conf
@@ -15,3 +15,4 @@ BACKUP=RSYNC
 BACKUP_URL=rsync://root@server/export/archives
 SSH_ROOT_PASSWORD="vagrant"
 USE_STATIC_NETWORKING=y
+KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"

rear-configs/local-with-sshfs.conf

@@ -1,5 +1,5 @@
 # The rear configuration file required during the workshop
-# at OSBconf 2016 - lab backup with sshfs
+# lab backup with sshfs
 # Actions to be taken:
 # 1/ copy this config to /etc/rear/local.conf
 # 2/ use this config file to run 'rear -v mkbackup'
@@ -15,3 +15,4 @@ ISO_DEFAULT=manual
 BACKUP=NETFS
 BACKUP_URL=sshfs://root@server/export/archives
 USE_STATIC_NETWORKING=y
+KERNEL_CMDLINE="$KERNEL_CMDLINE net.ifnames=0"

rear-configs/rear-workshop.spec

@@ -7,7 +7,7 @@ Summary:	Rear configuration files
 
 Group:		Applications/File
 License:	GPLv3
-URL:		https://github.com/rear/rear-workshop-osbconf-2016
+URL:		https://github.com/rear/rear-workshop
 Source0:	https://build.opensuse.org/package/show/home:gdha/rear-workshop/rear-workshop.tar.gz
 BuildRoot:	%(mktemp -ud %{_tmppath}/%{name}-%{version}-XXXXXX)