Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for Elastic Stack 6.x (elastic#165)
* Add 6.x versions to the template This commit adds the 6.x versions of Elasticsearch to the template. A unqiue bootstrap password is generated within the template and passed to Elasticsearch setup script to set the bootstrap.password in the keystore. Inverted all 5.x Elasticsearch version checks to use 2.x, since commands are the same across 5.x and and 6.x. * Rename INSTALL_PLUGINS to INSTALL_XPACK This commit renames INSTALL_PLUGINS variable to INSTALL_XPACK. The commercial plugins are separate in 2.x but bundled as one X-Pack plugin in 5.x and 6.x. Since the majority of installations now install with the one X-Pack plugin, rename the variable to reflect this. * Remove kibana4_server generated password This commit better aligns the es_kibana user in 2.x and the built-in kibana user in 5.x+. In 2.x, the es_kibana user is now assigned the kibana4_server role and the kibana4 role is removed. Kibana will use the es_kibana user credentials when interacting with Elasticsearch. In 5.x+, the kibana built-in user will use the password supplied for the kibana user and the es_kibana user is removed. Kibana will use the kibana user credentials when interacting with Elasticsearch. This change simplifies the template and a user will now always know the credentials used by Kibana. Closes elastic#32 * Allow bootstrap.password to be supplied This commit adds a parameter to allow a user to set the bootstrap.password added to the keystore to set up built-in users. It can be useful to know this value in the event one needs to re-set up built-in users, and it avoids the need to log in to each node to set it to a known value. If not supplied, a unique value is generated within the ARM template, using the uniqueString() function seeded with the resourceGroup id, deployment name and securityAdminPassword. * Add logstash_system user password parameter This commit adds a parameter for the logstash_system built-in user account in Elasticsearch 5.2.0+. Rename all remnants of kibana4 to simply kibana. * Add mount options and remove sudo calls in disk partitioning Add support to change mount options. Include nofail into default mount options as recommended in https://docs.microsoft.com/en-us/azure/virtual-machines/linux/add-disk Remove sudo calls within the script. Must be run as root already and using sudo does not transfer environment variables. * Increase retries to 60 Increase the number of retries made to see when Elasticsearch is up to 60, so that we wait a total of up to 5 minutes for the node to come up. * Comment out conf-enabled include in monit configuration conf-enabled include is not needed and contains no files, so comment out from monit configuration to suppress the superfluous warning written to stderr. * Write log files to /var/log/elasticsearch When unspecified, Elasticsearch 6.x will attempt to write log files to /usr/share/elasticsearch/logs which the elasticsearch user under which the service runs does not have access to do. This can be observed in the monit log with cat /var/log/monit.log which looks similar to [UTC Feb 12 22:37:07] info : New Monit id: 4c9c50e3bff3365388e922d50b305764 Stored in '/var/lib/monit/id' [UTC Feb 12 22:37:07] info : Starting Monit 5.16 daemon [UTC Feb 12 22:37:07] info : 'data-0' Monit 5.16 started [UTC Feb 12 22:37:15] info : Reinitializing monit daemon [UTC Feb 12 22:37:15] info : Awakened by the SIGHUP signal Reinitializing Monit - Control file '/etc/monit/monitrc' [UTC Feb 12 22:37:15] error : Cannot create socket to [localhost]:2812 -- Connection refused [UTC Feb 12 22:37:15] warning : /etc/monit/monitrc:290: Include failed -- Success '/etc/monit/conf-enabled/*' [UTC Feb 12 22:37:15] info : Starting Monit HTTP server at [localhost]:2812 [UTC Feb 12 22:37:15] info : Monit HTTP server started [UTC Feb 12 22:37:15] info : 'data-0' Monit reloaded [UTC Feb 12 22:37:15] error : 'elasticsearch' process is not running [UTC Feb 12 22:37:15] info : 'elasticsearch' trying to restart [UTC Feb 12 22:37:15] info : 'elasticsearch' start: /etc/init.d/elasticsearch [UTC Feb 12 22:37:45] error : 'elasticsearch' failed to start (exit status 1) -- /etc/init.d/elasticsearch: * Starting Elasticsearch Server 2018-02-12 22:37:17,914 main ERROR Unable to create file /usr/share/elasticsearch/logs/russ-test-6x.log java.io.IOException: Could not create directory /usr/share/elasticsearch/logs at org.apach [UTC Feb 12 22:38:15] error : 'elasticsearch' process is not running [UTC Feb 12 22:38:15] info : 'elasticsearch' trying to restart [UTC Feb 12 22:38:15] info : 'elasticsearch' start: /etc/init.d/elasticsearch [UTC Feb 12 22:38:46] error : 'elasticsearch' failed to start (exit status 1) -- /etc/init.d/elasticsearch: * Starting Elasticsearch Server 2018-02-12 22:38:18,292 main ERROR Unable to create file /usr/share/elasticsearch/logs/russ-test-6x.log java.io.IOException: Could not create directory /usr/share/elasticsearch/logs at org.apach Explicitly set path.logs to /var/log/elasticsearch.log. It may be useful set the location to a datadisk, if available, in future. * Always specify Content-Type in curl requests * Remove all usage of sudo Script runs elevated already. * Disallow Standard_A0 and Standard_A1 SKUs for Kibana VM The Kibana process that optimizes and caches browser bundles for plugins and on startup currently requires > 1.5Gb RAM. For 6.x, a Standard_A1 SKU VM has been found to have insufficient RAM and CPU for the optimizer process of Kibana X-Pack. * Update 14.04 SKU for Elasticsearch 2.x * Remove unreachable branch Kibana 5.x+ installations use the deb package rather than the tar archive, so remove unreachable branch in old_download_unzip_kibana() function * Set path.data for non-data nodes When unspecified, Elasticsearch 6.x will attempt to write data to /usr/share/elasticsearch/data which the elasticsearch user under which the service runs does not have access to. This can be seen in the elasticsearch.log as similar to [2018-02-13T05:19:08,503][ERROR][o.e.b.Bootstrap ] Exception java.lang.IllegalStateException: Unable to access 'path.data' (/usr/share/elasticsearch/data) at org.elasticsearch.bootstrap.FilePermissionUtils.addDirectoryPath(FilePermissionUtils.java:70) ~[elasticsearch-6.2.1.jar:6.2.1] at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:307) ~[elasticsearch-6.2.1.jar:6.2.1] at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:262) ~[elasticsearch-6.2.1.jar:6.2.1] ... * Update README This commit adds the new parameters introduced to support 6.x. Tidy up descriptions for existing parameters. * Fixed 'npm run test-deploy' and 'npm run test' to account for new parameters * Check the elastic user exists This commit guards against the case where the bootstrap password is the same as the elastic superuser password for 5.x and 6.x clusters. In this scenario, the .security index should also be checked to ensure the elastic superuser has been created. The _xpack/security/user/elastic endpoint cannot be used for this because it may return details related to the bootstrap user. * Remove EOL'ed 2.x versions This commit removes the ability to deploy 2.x versions of Elasticsearch and associated Kibana that are no longer supported by Elastic based on: https://www.elastic.co/support/eol * Remove java8 apt package patching oracle-java8-installer apt package has now been updated to use 161: https://launchpad.net/~webupd8team/+archive/ubuntu/java Remove the patching of the apt-package and rely on the package as is
- Loading branch information
