Skip to content

Commit

Permalink
alertmanager up and running
Browse files Browse the repository at this point in the history
  • Loading branch information
tboeghk committed May 26, 2023
1 parent 937e168 commit de62441
Show file tree
Hide file tree
Showing 6 changed files with 20 additions and 6 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
**/secrets
**/.env

# generated files
prometheus-grafana/rootfs/etc/prometheus.yaml
Binary file added images/grafana-alertmanager.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/grafana-prometheus-performance.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
21 changes: 16 additions & 5 deletions prometheus-grafana/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ without making them accessible to Grafana Cloud.
> 💡 This deplyoment pattern is common to overcome the rather limited
> free metrics volume in Grafana Cloud
<img src="../images/grafana-configure-prometheus.jpg" width="400" style="float: right; margin-left: 1em;">

### Launch a local Prometheus

Expand All @@ -79,21 +78,23 @@ he discovered out of the box.

### Configure Grafana Cloud

<img src="../images/grafana-node-exporter-dashboard.jpg" width="400" style="float: right; margin-left: 1em;">

[Log into your Grafana cloud account](https://grafana.com/auth/sign-in)
and launch Grafana.

#### Configure Prometheus Data Source

<img src="../images/grafana-prometheus-performance.png" width="300" style="float: right; margin-left: 1em;">
<img src="../images/grafana-configure-prometheus.jpg" width="400">

Go to `Connections -> Data Sources` and add a new Prometheus Data Source.
Make it default and use the url `https://prometheus.PETNAME.workshop.o11ystack.org`.
Under _Performance_ select the most recent Prometheus version.

> You can verify that your metrics are available in Grafana using the `Explore` section.
#### Import Dashboards into Grafana Cloud

<img src="../images/grafana-node-exporter-dashboard.jpg" width="250" style="float: right; margin-left: 1em;">

In Grafana, import the following Dashboards using their Grafana Cloud ID

Expand All @@ -105,16 +106,26 @@ In Grafana, import the following Dashboards using their Grafana Cloud ID

## 🚨 Alerts

<img src="../images/grafana-alertmanager.png" width="400" style="float: right; margin-left: 1em;">

```
docker-compose -f docker-compose-alerts.yaml up
docker-compose -f docker-compose-alerts.yaml up -d
```

```bash
set -a; source .env; set +a
envsubst < rootfs/etc/prometheus/prometheus.alertmannager.yaml.template \
envsubst < rootfs/etc/prometheus/prometheus.alertmanager.yaml.template \
>> rootfs/etc/prometheus/prometheus.yaml
```

Restart your Prometheus instance

In Grafana.com, go to `Connections -> Data Sources` and add a new Alertmanager Data Source.
Make it default and use the url `https://alertmanager.PETNAME.workshop.o11ystack.org`. Make
sure to set the _Prometheus_ implementation.

In Grafana.com, you can now browse your Prometheus alerts `Alerts & IRM > Alerting > Alert Rules`.


## 🪵 Log Management

Expand Down
2 changes: 1 addition & 1 deletion prometheus-grafana/docker-compose-alerts.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: '2.4'

services:
prometheus:
alertmanager:
image: prom/alertmanager:v0.25.0
command: |
--config.file=/etc/prometheus/alertmanager.yaml
Expand Down

0 comments on commit de62441

Please sign in to comment.