Add PC5.7.1, make IPs relative, formatting, change to password to env…

…ironment variable

.gitignore

@@ -2,3 +2,5 @@ example_pocs.txt
 cache/
 logs/
 .DS_Store
+.kitchen/
+.kitchen.local.yml

bugs.txt

@@ -1,5 +1,43 @@
+- Move AutoDC to DHCP? and adjust DNS for SRE HPOC subnets?
 - PC 5.6:
-  - Revalidate it works
+  - Revalidate it works, add AOS 5.5 dependency note
+  - 5.6.1 http://download.nutanix.com/pc/one-click-pc-deployment/5.6.1/v1/euphrates-5.6.1-stable-prism_central_metadata.json
+- bugs from [email protected]:
+  - test cases: external URLs working (PC x, sshpass, jq, autodc, etc.)
+  - i created the cluster file and run it with stage_Workshop and it gives me the menu
+but none of the inputs are accepted:
+    - ./stage_Workshop.sh
+    - example_poc.txt
+    - 2 (no response, back to menu)
+    " you can login and see PE tasks while in flight, but that’s not as informative"
+    - verify doesn't do much
+    - Test 10/10|Substate=running, exec properly, sleep 7 seconds..
+    Error 12: AutoDC VM running, giving up after 11 tries.
+    AutoDC pending
+    AutoDC complete
+      - You can remediate by running it again or by deleting the AutoDC VM from PE and rerunning the script. I should make it try more times, perhaps.
+    - PC_Configure|OPTIONAL: send bin to PC, sshpass: no such file or dir _TEST 3/3 Error 11 giving up after 3 tries...
+- SRE Clusters of HPOC (10.63.x.x)
+  - Cluster IP: https://10.63.30.150:9440/console/#login
+    Prism UI Credentials: admin/nx2Tech975!
+    CVM Credentials: nutanix/nx2Tech975!
+    AHV Host Credentials: root / nx2Tech975!
+
+    AOS Version: 5.6
+    Hypervisor Version: AHV 20170830.115 (AOS5.6+)
+
+    NETWORK INFORMATION
+    Subnet Mask: 255.255.252.0
+    Gateway: 10.63.28.1
+    Nameserver IP: 10.63.25.10
+
+    SECONDARY NETWORK INFORMATION
+    Secondary VLAN: 0
+    Secondary Subnet: 255.255.252.0
+    Secondary Gateway: 10.63.28.1
+    Secondary IP Range: 10.63.31.146-149
+
+
 + PC 5.7.0.1:
   - document public cloud account/credentials
     - Add to: https://drt-it-github-prod-1.eng.nutanix.com/akim-sissaoui/calm_aws_setup_blueprint/blob/master/Action%20Create%20Project/3-Create%20AWS%20Calm%20Entry
@@ -8,7 +46,6 @@
     - PE, PC: use RBAC user for APIs, etc.: cluster Admin
     - improve/run poc_samba_users.sh
   - nuclei (run local from container?)
-    - http://vcdx56.com/2017/08/change-nutanix-prism-ui-login-screen/ PC UI customization
     - version.get # gives API 3.1 and AOS 5.7.0.1 (bug!)
       - vs: cat /etc/nutanix/release_version
     - project.create name=mark.lavi.test \
@@ -57,41 +94,52 @@
   - OPTIMIZATION: Upload AutoDC image in parallel with PC.tar
 - Overall TODO:
   - Calm 5.7 bootcamp labs and 5.5-6 bugs
+    - https://github.com/nutanixworkshops/introcalm
+    vs. https://github.com/mlavi/calm_workshop
   - file Calm bugs from guide
   - Refactor 10.21 out further!
   - create cache, use cache, propagate cache to PC, fall back to global
   - review, refactor & migrate to bugs.txt: TODO, TOFIX comments
   - refactor out all passwords, hardcoded values to variables
     - SSP Admins
   - Insure exit codes unique/consistent, error messages consistent
-  - Bash test framework for unit tests:
-    - BATS
+  - Bash test framework for unit tests and on blueprints?
+    - https://kitchen.ci/ which can do spec, BATS, etc. = https://github.com/test-kitchen/test-kitchen
+      - https://kitchen.ci/docs/getting-started/writing-test
+      - https://serverspec.org/ DSL Spec TDD
+      - http://rspec.info/ Ruby TDD
+      - inspec
+        - more compliance from supermarket
+        - https://dev-sec.io/features.html#os-hardening
+        - https://www.cisecurity.org/cis-benchmarks/
+      - https://en.wikipedia.org/wiki/ERuby
+      - https://www.engineyard.com/blog/bats-test-command-line-tools
+      - https://medium.com/@pimterry/testing-your-shell-scripts-with-bats-abfca9bdc5b9
+        - http://ohmyz.sh/
+        - https://github.com/jakubroztocil/httpie#scripting
+        - https://github.com/pimterry/git-confirm
+    - BATS https://github.com/bats-core/bats-core
     - https://invent.life/project/bash-infinity-framework
     - Runit/rundeck? http://bashdb.sourceforge.net/
     - Tests:
       - userX login to PE, PC
       - userX new project, upload, run blueprint
-      - rspec, serverspec on blueprint?
-  - Atom.io packages added:
-    - npm i -g bash-language-server ; https://atom.io/packages/ide-bash
-    - https://atom.io/packages/atom-ide-ui
-      - Disabled linter & linter-ui-default
-    - Interesting:
-      - https://atom.io/packages/script
-      - https://atom.io/packages/platformio-ide-terminal
+  - Base images/boxes: https://github.com/chef/bento
+    - https://qemu.weilnetz.de/doc/qemu-doc.html#disk_005fimages_005fssh
+      qemu-system-x86_64 -drive file=ssh://[user@]server[:port]/path[?host_key_check=host_key_check]
 
   + clear; MY_PE_PASSWORD='nx2Tech!'; sshpass -p "${MY_PE_PASSWORD}" \
   ssh -o StrictHostKeyChecking=no -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null \
   [email protected].${MY_HPOC_NUMBER}.37 'pkill tail; ps -efww|grep calm'
 + AutoDC:
   + NTNXLAB, ntnxlab.local, root:nutanix/4u
   + samba --version Version 4.2.14-Debian
+  - https://gitlab.com/mlavi/alpine-dc
 + PE:
   + [email protected] auth test fine
 
 Citations:
 - https://drt-it-github-prod-1.eng.nutanix.com/sylvain-huguet/auto-hpoc
--
 
 Push button Calm:
 - https://github.com/mlavi/stageworkshop/blob/master/guidebook.md

chefignore

@@ -0,0 +1 @@
+.kitchen

gemfile

@@ -0,0 +1,12 @@
+# $ gem install bundler# 1.16.2  # https://bundler.io/
+# $ bundle install
+source 'https://rubygems.org'
+gem 'bundler', '>= 1.16.1'
+gem 'test-kitchen'
+gem 'kitchen-vagrant'
+gem 'serverspec'
+gem 'inspec'
+gem 'kitchen-inspec'
+gem 'berkshelf'
+# The `minitar` executable is no longer bundled with `minitar`. If you are
+# expecting this executable, make sure you also install `minitar-cli`.

kitchen.yml

@@ -0,0 +1,15 @@
+---
+driver:
+  name: vagrant
+
+provisioner:
+  name: chef_solo
+
+platforms:
+  - name: ubuntu-16.04
+  - name: centos-7
+
+suites:
+  - name: default
+    run_list:
+    attributes:

scripts/common.lib.sh

@@ -1,6 +1,5 @@
 #!/usr/bin/env bash
 
-# TODO: lost local override for verbose
      CURL_OPTS='--insecure --silent --show-error' # --verbose'
 CURL_POST_OPTS="${CURL_OPTS} --max-time 5 --header Content-Type:application/json --header Accept:application/json --output /dev/null"
 CURL_HTTP_OPTS="${CURL_POST_OPTS} --write-out %{http_code}"
@@ -20,14 +19,6 @@ function CheckArgsExist {
       exit -1
     fi
   done
-
-  # if [[ -z ${MY_HPOC_NUMBER} ]]; then
-  #   # Derive HPOC number from IP 3rd byte
-  #   #MY_CVM_IP=$(ip addr | grep inet | cut -d ' ' -f 6 | grep ^10.21 | head -n 1)
-  #   MY_CVM_IP=$(/sbin/ifconfig eth0 | grep 'inet ' | awk '{ print $2}')
-  #   array=(${MY_CVM_IP//./ })
-  #   MY_HPOC_NUMBER=${array[2]}
-  # fi
 }
 
 function Download {
@@ -47,7 +38,7 @@ function Download {
     local _OUTPUT=''
     curl ${CURL_OPTS} ${_HTTP_RANGE_ENABLED} --remote-name --location ${1}
     _OUTPUT=$?
-    DEBUG=1; if [[ ${DEBUG} ]]; then log "DEBUG: curl exited ${_OUTPUT}."; fi
+    #DEBUG=1; if [[ ${DEBUG} ]]; then log "DEBUG: curl exited ${_OUTPUT}."; fi
 
     if (( ${_OUTPUT} == 0 )); then
       log "Success: ${1##*/}"
@@ -77,28 +68,20 @@ function remote_exec { # TODO: similaries to Check_Prism_API_Up
   local  _ACCOUNT='nutanix'
   local _ATTEMPTS=3
   local    _ERROR=99
-  local     _HOST="${MY_PE_HOST}"
+  local     _HOST=${MY_PE_HOST}
   local     _LOOP=0
   local _PASSWORD="${MY_PE_PASSWORD}"
   local    _SLEEP=${SLEEP}
   local     _TEST=0
 
   case ${2} in
-    'PE' )
-      if [[ -z ${MY_PE_HOST} ]]; then
-        _HOST=localhost
-      fi
-      ;;
     'PC' )
-      if [[ -z ${MY_PC_HOST} ]]; then
-        #_HOST=localhost
-        MY_PC_HOST=$(echo ${MY_PE_HOST} | sed s/7$/9/)
-      fi
+          _HOST=${MY_PC_HOST}
       _PASSWORD='nutanix/4u' # TODO: hardcoded p/w
       ;;
     'LDAP_SERVER' )
        _ACCOUNT='root'
-          _HOST=$(echo ${MY_PE_HOST} | sed s/37$/40/)
+          _HOST=${LDAP_HOST}
       _PASSWORD='nutanix/4u' # TODO: hardcoded p/w
          _SLEEP=7
       ;;
@@ -114,12 +97,12 @@ function remote_exec { # TODO: similaries to Check_Prism_API_Up
     case "${1}" in
       'SSH' | 'ssh')
        #DEBUG=1; if [[ ${DEBUG} ]]; then log "_TEST will perform ${_ACCOUNT}@${_HOST} ${3}..."; fi
-        sshpass -p ${_PASSWORD} ssh -x ${SSH_OPTS} ${_ACCOUNT}@${_HOST} "${3}"
+        SSHPASS="${_PASSWORD}" sshpass -e ssh -x ${SSH_OPTS} ${_ACCOUNT}@${_HOST} "${3}"
         _TEST=$?
         ;;
       'SCP' | 'scp')
         #DEBUG=1; if [[ ${DEBUG} ]]; then log "_TEST will perform scp ${3} ${_ACCOUNT}@${_HOST}:"; fi
-        sshpass -p ${_PASSWORD} scp ${SSH_OPTS} ${3} ${_ACCOUNT}@${_HOST}:
+        SSHPASS="${_PASSWORD}" sshpass -e scp ${SSH_OPTS} ${3} ${_ACCOUNT}@${_HOST}:
         _TEST=$?
         ;;
       *)
@@ -166,7 +149,7 @@ function Dependencies {
       export PATH=${PATH}:${HOME}
 
       if [[ `uname --operating-system` == "GNU/Linux" ]]; then
-        # probably on NTNX CVM or PCVM = CentOS7
+        # TOFIX: assumption, probably on NTNX CVM or PCVM = CentOS7
         case "${2}" in
           sshpass )
             if [[ -z `which ${2}` ]]; then
@@ -254,15 +237,14 @@ function Check_Prism_API_Up { # TODO: similaries to remote_exec
 # Argument ${2} = OPTIONAL: number of attempts
 # Argument ${3} = OPTIONAL: number of seconds per cycle
   local _ATTEMPTS=${ATTEMPTS}
-  local     _HOST="${MY_PE_HOST}"
+  local     _HOST=${MY_PE_HOST}
   local     _LOOP=0
   local _PASSWORD="${MY_PE_PASSWORD}"
   local    _SLEEP=${SLEEP}
   local     _TEST=0
 
   if [[ ${1} == 'PC' ]]; then
         _HOST=${MY_PC_HOST}
-    #_PASSWORD='nutanix/4u' # TODO: hardcoded p/w
   fi
 
   if [[ ! -z ${2} ]]; then
@@ -280,8 +262,9 @@ function Check_Prism_API_Up { # TODO: similaries to remote_exec
       | tr -d \") # wonderful addition of "" around HTTP status code by cURL
 
     if (( ${_TEST} == 401 )) && [[ ${1} == 'PC' ]]; then
-      _PASSWORD='Nutanix/4u'
+      _PASSWORD='Nutanix/4u' # TODO: hardcoded p/w
       log "@${1}: Fallback: try initial password next cycle..."
+      break
     fi
 
     if (( ${_TEST} == 200 )); then

scripts/stage_calmhow.sh

@@ -38,7 +38,7 @@ function PC_Download
 
 function PE_Init
 {
-  local _DATA_SERVICE_IP=${HPOC_PREFIX}.38
+  local _DATA_SERVICE_IP=${HPOC_PREFIX}.$(($OCTET4 + 1))
 
   if [[ `ncli cluster get-params | grep 'External Data' | \
          awk -F: '{print $2}' | tr -d '[:space:]'` == "${_DATA_SERVICE_IP}" ]]; then
@@ -66,7 +66,7 @@ function PE_Init
     # Set external IP address:
     #ncli cluster edit-params external-ip-address=${MY_PE_HOST}
 
-    log "Set Data Services IP address to ${_DATA_SERVICE_I}P"
+    log "Set Data Services IP address to ${_DATA_SERVICE_IP}"
     ncli cluster edit-params external-data-services-ip-address=${_DATA_SERVICE_IP}
   fi
 }
@@ -335,7 +335,6 @@ function PC_Init
   if (( $? == 0 )) ; then
     log "IDEMPOTENCY: PC API responds, skip."
   else
-    MY_PC_HOST=${HPOC_PREFIX}.39
     log "Get NET_UUID,MY_CONTAINER_UUID from cluster: PC_Init dependency."
     MY_NET_UUID=$(acli "net.get ${MY_PRIMARY_NET_NAME}" | grep "uuid" | cut -f 2 -d ':' | xargs)
     log "${MY_PRIMARY_NET_NAME} UUID is ${MY_NET_UUID}"
@@ -412,15 +411,18 @@ function PC_Configure {
   # Execute that file asynchroneously remotely (script keeps running on CVM in the background)
   log "Launch PC configuration script"
   remote_exec 'ssh' 'PC' \
-   "LDAP_SERVER=${LDAP_SERVER} LDAP_HOST=${LDAP_HOST} MY_DOMAIN_FQDN=${MY_DOMAIN_FQDN} MY_DOMAIN_USER=${MY_DOMAIN_USER} MY_DOMAIN_PASS=${MY_DOMAIN_PASS} MY_PE_PASSWORD=${MY_PE_PASSWORD} MY_PC_VERSION=${MY_PC_VERSION} nohup bash /home/nutanix/stage_calmhow_pc.sh >> stage_calmhow_pc.log 2>&1 &"
+    "LDAP_SERVER=${LDAP_SERVER} LDAP_HOST=${LDAP_HOST} MY_DOMAIN_FQDN=${MY_DOMAIN_FQDN} \
+    MY_DOMAIN_USER=${MY_DOMAIN_USER} MY_DOMAIN_PASS=${MY_DOMAIN_PASS} \
+    MY_PC_HOST=${MY_PC_HOST} MY_PE_PASSWORD=${MY_PE_PASSWORD} MY_PC_VERSION=${MY_PC_VERSION} \
+    nohup bash /home/nutanix/stage_calmhow_pc.sh >> stage_calmhow_pc.log 2>&1 &"
   log "PC Configuration complete: try Validate Staged Clusters now."
 }
 
 #__main()__________
 
 # Source Nutanix environments (for PATH and other things)
 . /etc/profile.d/nutanix_env.sh
-. common.lib.sh # source common routines, global variables
+. common.lib.sh # source common routines, additional global variables
 
 log `basename "$0"`": PID=$$"
 
@@ -430,14 +432,16 @@ array=(${MY_PE_HOST//./ })
      OCTET1=${array[0]}
      OCTET2=${array[1]}
      OCTET3=${array[2]}
+     OCTET4=${array[3]}
 HPOC_PREFIX=${OCTET1}.${OCTET2}.${OCTET3}
+ MY_PC_HOST=${HPOC_PREFIX}.$(($OCTET4 + 2))
 
 MY_SP_NAME='SP01'
 MY_CONTAINER_NAME='Default'
 MY_IMG_CONTAINER_NAME='Images'
 
    LDAP_SERVER='AutoDC'
-     LDAP_HOST=${HPOC_PREFIX}.40
+     LDAP_HOST=${HPOC_PREFIX}.$(($OCTET4 + 3))
  MY_DOMAIN_URL="ldaps://${LDAP_HOST}/"
 MY_DOMAIN_FQDN='ntnxlab.local'
 MY_DOMAIN_NAME='NTNXLAB'
@@ -455,9 +459,12 @@ case ${MY_PC_VERSION} in
   5.6 )
     MY_PC_META_URL='http://10.21.250.221/images/ahv/techsummit/euphrates-5.6-stable-prism_central_metadata.json'
     ;;
-  5.7 | 5.7.0.1 )
+  5.7.0.1 )
     MY_PC_META_URL='http://download.nutanix.com/pc/one-click-pc-deployment/5.7.0.1/v1/pc-5.7.0.1-stable-prism_central_metadata.json'
     ;;
+  5.7 | 5.7.1 )
+    MY_PC_META_URL='http://10.21.249.53/pc-5.7.1-stable-prism_central_metadata.json'
+    ;;
   *)
     log "Errror: unsupported MY_PC_VERSION=${MY_PC_VERSION}!"
     log 'Browse to https://portal.nutanix.com/#/page/releases/prismDetails'
@@ -492,8 +499,6 @@ if (( $? == 0 )) ; then
   log "PC Configuration complete: Waiting for deployment to complete, API up..."
   log "$0: main: done!_____________________"
   echo
-  #log "Watching logs on PC..."
-  #BUG: Dependencies removed! remote_exec 'ssh' 'PC' "tail -f stage_calmhow_pc.log"
 else
   log "Error in main functional chain, exit!"
   exit 18