Upload Test Data

[jacderida]

• 1fe0ef9 chore: create node image with filebeat installed

  The installation of Filebeat during the deploy process was causing too much grief. Almost 100% of
  the time, it would fail because the unattended upgrades service was running in the background and
  preventing access to the package manager. It was also very slow to run.

  The filebeat role is split in two: one for installation and one for config. The installation role
  is applied to the node image, and the config role is applied during deploy.

• fe111df fix: return public ip for droplets

  For some reason, the previous implementation for listing droplets didn't take into account the fact
  that both public and private IPs can be returned. This was sometimes causing the private IP to be
  selected for the Logstash server.

  A new test case was added to explicitly select the public address.

• f535bc8 chore: new process for generating certificates

  After launching a new deployment of the Logstash server, Filebeat started complaining that IP SAN
  information was missing from the certificates we were using. The certificate generation process is
  now updated to include the IP information of the Logstash server.

• 7567b52 chore: cache the deployment inventory

  Serialize the deployment inventory report to JSON and cache it in a file.

  This is useful for quickly printing the inventory again, and it will also be used to quickly
  retrieve the peer list for use with commands related to test data.

• d081540 feat: upload test data command

  Provide a command to upload a set of standard test data, which is an archive with about 100mb of
  jpeg images. The process is roughly:

  • Run the deploy command
  • The client binary is now built along with the node and faucet then uploaded to S3
  • The rest of the testnet spins up as usual
  • Retrieve some tokens from the faucet
  • Run the upload-test-data command
  • The test data archive is downloaded from S3 and extracted
  • The safe client build is downloaded from S3
  • We use the files upload command to upload each file in the archive

  After all that, you get a list of the files and their addresses, with the potential to share those.
  Though the same list will also be made available through the safe client.