AWS Cloudwatch can be used to monitor logs and metrics. To use AWS Cloudwatch with the Boomi runtimes there needs to be two agents involved. First, the amazon-cloudwatch-agent will be used to send data to Cloudwatch and the second agent will be collectd, which will be used to retrieve JMX properties from the Boomi runtime.
The CloudWatch Agent uses a policy that is attached to a role, and then attached to the EC2 instance. CloudWatchAgentAdminPolicy is the AWS managed policy that will be used. A copy of the policy is enclosed within the repo, amazon-cloudwatch-agent.json.
collectd is a powerful and robust service that is used to collect metrics. It’s built on C and uses plug-ins for it’s functionality. We’ll be covering the plug-ins for logging of collectd, java, and jmx.
First we will want to install a few packages. I’ll be assuming RHEL as the linux distribution.
# Install the Amazon Linux Extras package within Amazon Linux 2
sudo yum install -y amazon-linux-extras
# Install a local version of java.
# You can use the Boomi's runtime version of Java as an additional option.
amazon-linux-extras install -y java-11-openjdk-devel
# Install collectd and additional packages
amazon-linux-extras install -y collectd
sudo yum install -y collectd-java
sudo yum install -y collectd-generic-jmx
Once, that is installed, next create a symlink for libjvm.so. The documentation uses libjvm.so from the java-11-openjdk-devel package. A symlink to java-11-openjdk is used so that the libjvm.so symlink doesn't break when java is updated. collectd is assuming that it is located within /usr/lib64/ and will fail to load it is not found. There are other ways to achieve this same configuration but I found this to be the easiest to implement.
# This directory is from the java-11-openjdk-devel package
# Use this location for libjvm.so because it uses a symlink (/usr/lib/jvm/java-11/openjdk)
# /usr/lib/jvm/java-11-openjdk/lib/server/libjvm.so
libjvm_location=/usr/lib/jvm/java-11-openjdk/lib/server/libjvm.so
ln -s $libjvm_location /usr/lib64/libjvm.so
Next, modify the collectd.conf file and overwrite what is currently within /etc/collectd.conf.
Within collectd.conf there are 4 plugins being used. logfile will write the collectd logs to /var/log/collectd.log. This file can be helpful to troubleshoot issues. The LogLevel can be changed to info, which will populate the log file with metrics. The conf file below has it set to warning to limit the amount of data within the file. The write_log plugin defines the format that data is written to in the collectd.log file.
The java plugin has a java and a GenericJMX plugin within. They are used to monitor the JMX properties. The connection section defines how to connect to the Boomi runtime.
Finally, the network plugin acts as a server for the Amazon CloudWatch Agent to listen to for metrics.
collectd has now been setup.
# Install Amazon CloudWatch Agent
sudo yum install -y amazon-cloudwatch-agent
amazon-cloudwatch-agent will get installed to /opt/aws/amazon-cloudwatch-agent/bin. There are a few ways to start amazon-cloudwatch-agent. bin/amazon-cloudwatch-agent-config-wizard will start the auto config. This is helpful if you do not know what you want.
For us, we are going to use the config file that is below and implement it by running the following command. The user that the server is being run as is defined. Additional, under the collectd metrics, the connection that was referenced earlier under network is defined.
# Set the CloudWatch Agent's directory
AWS_CLOUDWATCH_AGENT_HOME="/opt/aws/amazon-cloudwatch-agent/bin"
# Create or overwrite what is in the amazon-cloudwatch-agent.json
# with what is below
sudo vi $AWS_CLOUDWATCH_AGENT_HOME/amazon-cloudwatch-agent.json
$AWS_CLOUDWATCH_AGENT_HOME/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:$AWS_CLOUDWATCH_AGENT_HOME/amazon-cloudwatch-agent.json
Used the amazon-cloudwatch-agent.json.
The previous sections outline the manual steps. The aws-cloudwatch-boomi-monitor.sh script can automate those steps. Place the bash script, collectd.conf, and amazon-cloudwatch-agent.json files into the same directory by cloning the repo. Then execute the aws-cloudwatch-boomi-monitor.sh bash script.
Go to Cloudwatch and use the following path to start monitoring the JMX metrics. Metrics -> All Metrics -> Browse -> CWAgent -> ImageId, InstanceId, InstanceType, instance, type, type_instance (there are multiple that look similar)
There are two log files that can be useful for determining errors.
# Amazon CloudWatch Logs
vi /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log
# collectd log file.
# Update LogLevel within collectd.conf to info to see more detail
vi /var/log/collectd.log
Installing CloudWatch Agent on EC2
How to Better Monitor Your Custom Application Metrics Using Amazon CloudWatch Agent