Skip to content

Commit

Permalink
docs: Fillout metrics section of our WAL docs (backport release-3.2.x) (
Browse files Browse the repository at this point in the history
#15837)

Co-authored-by: Dylan Guedes <[email protected]>
  • Loading branch information
loki-gh-app[bot] and DylanGuedes authored Jan 17, 2025
1 parent cfe16ad commit d276c61
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion docs/sources/operations/storage/wal.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,13 @@ The WAL also includes a backpressure mechanism to allow a large WAL to be replay

### Metrics

The following metrics are available for monitoring the WAL:

* `loki_ingester_wal_corruptions_total`: Total number of WAL corruptions encountered
* `loki_ingester_wal_disk_full_failures_total`: Total number of disk full failures
* `loki_ingester_wal_records_logged`: Counter for WAL records logged
* `loki_ingester_wal_logged_bytes_total`: Total bytes written to WAL

## Changes to deployment

1. Since ingesters need to have the same persistent volume across restarts/rollout, all the ingesters should be run on [StatefulSet](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/) with fixed volumes.
Expand Down Expand Up @@ -120,4 +127,4 @@ By following the above steps, you can ensure a smooth scaling down process for t

* When the ingester restarts for any reason (upgrade, crash, etc), it should be able to attach to the same volume in order to recover back the WAL and tokens.
* 2 ingesters should not be working with the same volume/directory for the WAL.
* A Rollout should bring down an ingester completely and then start the new ingester, not the other way around.
* A rollout should bring down an ingester completely and then start the new ingester, not the other way around.

0 comments on commit d276c61

Please sign in to comment.