Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating ECS Fargate/EC2 Documentation #2859

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

mrethers
Copy link

PR Description

Update to the ECS documentation to clarify options for Fargate and EC2 and collect application telemetry including metrics, traces and logs.

Notes to the Reviewer

PR Checklist

  • [ x] Documentation added

@CLAassistant
Copy link

CLAassistant commented Feb 26, 2025

CLA assistant check
All committers have signed the CLA.

@mrethers mrethers closed this Feb 26, 2025
@mrethers mrethers reopened this Feb 26, 2025
Copy link
Contributor

@clayton-cornell clayton-cornell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass with suggestions. I'd like to have another read through once these initial suggestions are reviewed/applied.


There are three different ways you can use {{< param "PRODUCT_NAME" >}} to collect Amazon ECS or AWS Fargate telemetry data.
Metrics are available from various sources including ECS itself, the ECS instances when using EC2, X-Ray and your own application. You can also collect logs and traces from your applications instrumented for Prometheus or OTLP.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Metrics are available from various sources including ECS itself, the ECS instances when using EC2, X-Ray and your own application. You can also collect logs and traces from your applications instrumented for Prometheus or OTLP.
Metrics are available from various sources including ECS itself, the ECS instances when using EC2, X-Ray, and your own application.
You can also collect logs and traces from your applications instrumented for Prometheus or OTLP.

Comment on lines +17 to +20
1. [Collecting task and container metrics](#collecting-task-and-container-metrics)
1. [Collecting application telemetry](#collecting-application-telemetry)
1. [Collecting EC2 instance metrics](#collecting-ec2-instance-metrics)
1. [Collecting application logs](#collecting-logs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. [Collecting task and container metrics](#collecting-task-and-container-metrics)
1. [Collecting application telemetry](#collecting-application-telemetry)
1. [Collecting EC2 instance metrics](#collecting-ec2-instance-metrics)
1. [Collecting application logs](#collecting-logs)
1. [Collect task and container metrics](#collect-task-and-container-metrics)
1. [Collect application telemetry](#collect-application-telemetry)
1. [Collect EC2 instance metrics](#collect-ec2-instance-metrics)
1. [Collect application logs](#collect-logs)

@@ -25,77 +26,157 @@ There are three different ways you can use {{< param "PRODUCT_NAME" >}} to colle
* Identify where {{< param "PRODUCT_NAME" >}} writes received telemetry data.
* Be familiar with the concept of [Components][] in {{< param "PRODUCT_NAME" >}}.

## Use a custom OpenTelemetry configuration file from the SSM Parameter store
## Collecting task and container metrics
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Collecting task and container metrics
## Collect task and container metrics

We should use bare infinitives, no gerunds... so in plain English... a simple task-based thing vs the "ing" form of the verb.


You can upload a custom OpenTelemetry configuration file to the SSM Parameter store and use {{< param "PRODUCT_NAME" >}} as a telemetry data collector.
In this configuration, an OTEL collector is added to the task running your application and uses the ECS Metadata Endpoint to gather task and container metrics in your cluster.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In this configuration, an OTEL collector is added to the task running your application and uses the ECS Metadata Endpoint to gather task and container metrics in your cluster.
In this configuration, you add an OTEL collector to the task running your application, and it uses the ECS Metadata Endpoint to gather task and container metrics in your cluster.

Making the sentence more active

You can configure the AWS Distro for OpenTelemetry Collector with the `AOT_CONFIG_CONTENT` environment variable.
This environment variable contains a full collector configuration file and it overrides the configuration file used in the collector entry point command.
In ECS, you can set the values of environment variables from AWS Systems Manager Parameters.
You can choose between two collector implementations:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit vague what the choice is here... at least with how the bullet points are worded.

Is it simply this?

  • You can use ADOT, the AWS OpenTelemetry collector.
  • You can use Alloy as a collector next to the Prometheus ECS collector.


Just like described in the [Configuring ADOT](#configuring-adot) section, you need to use a custom configuration SSM Parameter based on the [sample](https://github.com/aws-observability/aws-otel-collector/blob/main/config/ecs/otel-instance-metrics-config.yaml) config file in order to route the telemetry to your final destination.

## Collecting application telemetry
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Collecting application telemetry
## Collect application telemetry


SSH or connect to the Amazon ECS or AWS Fargate-managed container. Refer to [9 steps to SSH into an AWS Fargate managed container][steps] for more information about using SSH with Amazon ECS or AWS Fargate.
To collect metrics and traces emitted by your application, use the OTLP endpoints exposed by the collector side car container regardless of the collector implementation. Just specify `localhost` as the host name, which is default for most instrumentation library and agents.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
To collect metrics and traces emitted by your application, use the OTLP endpoints exposed by the collector side car container regardless of the collector implementation. Just specify `localhost` as the host name, which is default for most instrumentation library and agents.
To collect metrics and traces emitted by your application, use the OTLP endpoints exposed by the collector side car container regardless of the collector implementation.
Specify `localhost` as the host name, which is default for most instrumentation library and agents.


You can also use your own method to connect to the Amazon ECS or AWS Fargate-managed container as long as you can pass the parameters needed to install and configure {{< param "PRODUCT_NAME" >}}.
For prometheus endpoints, add a scrape job to the ADOT or Alloy config, and use `localhost` , service port and endpoint path.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For prometheus endpoints, add a scrape job to the ADOT or Alloy config, and use `localhost` , service port and endpoint path.
For Prometheus endpoints, add a scrape job to the ADOT or {{< param "PRODUCT_NAME" >}} configuration, and use `localhost` , service port, and endpoint path.


### Install {{% param "PRODUCT_NAME" %}}
## Collecting Logs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Collecting Logs
## Collect Logs


After connecting to your instance, follow the {{< param "PRODUCT_NAME" >}} [installation][install], [configuration][configure] and [deployment][deploy] instructions.
The easiest way to collect application logs in ECS is to leverage the [AWS firelens log driver][firelens-doc]. Depending on your use case, you can forward your logs to the collector container in your task using the [FluentBit plugin for OpenTelemetry][fluentbit-otel-plugin] or using the FluentBit Loki plugin. You can also send everything directly to your final destination.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The easiest way to collect application logs in ECS is to leverage the [AWS firelens log driver][firelens-doc]. Depending on your use case, you can forward your logs to the collector container in your task using the [FluentBit plugin for OpenTelemetry][fluentbit-otel-plugin] or using the FluentBit Loki plugin. You can also send everything directly to your final destination.
The easiest way to collect application logs in ECS is to leverage the [AWS firelens log driver][firelens-doc].
Depending on your use case, you can forward your logs to the collector container in your task using the [FluentBit plugin for OpenTelemetry][fluentbit-otel-plugin] or using the FluentBit Loki plugin.
You can also send everything directly to your final destination.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants