Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

As a Korifi API user I want to be able to get app/process metrics via the /api/v1/read endpoint #3752

Open
georgethebeatle opened this issue Jan 24, 2025 · 0 comments

Comments

@georgethebeatle
Copy link
Member

georgethebeatle commented Jan 24, 2025

Background

As of today Korifi implements metrics-fetching by directly querying the custom resources provided by the metrics server.

While this is enough to enable the basic cf push behaviour, it violates one of CF's abstractions that enables logs and metrics to be served by an external component that speaks a well-defined API.

Acceptance

GIVEN I have pushed an app to Korifi
WHEN I run cf curl /api/v1/read/<app-guid>?envelope_types=GAUGE
THEN I get a 403 Forbidden Response

WHEN I am bound to a role that is allowed to read cfapps
WHEN I run cf curl /api/v1/read/<app-guid>?envelope_types=GAUGE
THEN I get a resonse as follows

NOTE: this is the complete envelope as it is returned by classic cf. We should only support the minimal set of fields that enable basic cf commands (cf push, cf app)

"envelopes": {
  "batch": [
    {
      "timestamp": "1737732014790586000",
      "instance_id": "0",
      "deprecated_tags": {},
      "tags": {
        "app_id": "e75acbc9-fb11-4200-800b-bfcec928e758",
        "app_name": "dora",
        "deployment": "cf",
        "index": "41432433-3e0b-401b-a1b6-068708a4688a",
        "instance_id": "0",
        "ip": "10.244.0.139",
        "job": "diego-cell",
        "organization_id": "37707b21-ede7-4940-bb95-5724b714d4ae",
        "organization_name": "org",
        "origin": "rep",
        "process_id": "b7585d8a-ab70-4061-931b-278439a60f13",
        "process_instance_id": "85cb3c9f-87cd-4f2d-420f-ffc9",
        "process_type": "worker",
        "source_id": "e75acbc9-fb11-4200-800b-bfcec928e758",
        "space_id": "6cdfc65b-5b9d-43db-a7ce-2445996b1123",
        "space_name": "space"
      },
      "gauge": {
        "metrics": {
          "log_rate": {
            "unit": "B/s",
            "value": 0
          }
        },{
          "log_rate_limit": {
            "unit": "B/s",
            "value": -1
          }
        },{
          "container_age": {
            "unit": "nanoseconds",
            "value": 204978503253
          }
        },{
          "disk": {
            "unit": "bytes",
            "value": 136507392
          }
        },{
          "memory": {
            "unit": "bytes",
            "value": 47553649
          }
        },{
          "cpu": {
            "unit": "percentage",
            "value": 0.5353665896914775
          }
        },{
          "disk_quota": {
            "unit": "bytes",
            "value": 1073741824
          }
        },{
          "memory_quota": {
            "unit": "bytes",
            "value": 268435456
          }
        }
      }
    },
    ...
  ]
}

WHEN I run cf curl /api/v1/read/<app-guid>?envelope_types=GAUGE&start_time=<timestamp>
THEN I only get envelopes newer than

WHEN I run cf curl /api/v1/read/<app-guid>?envelope_types=GAUGE&limit=1
THEN I only get one envelope

WHEN I run cf curl /api/v1/read/<app-guid>?envelope_types=GAUGE&descending=true
THEN I get a the newest envelopes first

Dev Notes

Log Cache API

@github-project-automation github-project-automation bot moved this to 🧊 Icebox in Korifi - Backlog Jan 24, 2025
@georgethebeatle georgethebeatle changed the title Expose Korifi's metrics-fetching capabilities via the /api/v1/read endpoint of the Korifi API As a Korifi API user I want to be able to get app/process metrics via the /api/v1/read endpoint Jan 24, 2025
@georgethebeatle georgethebeatle moved this from 🧊 Icebox to 🇪🇺 To do in Korifi - Backlog Jan 24, 2025
@georgethebeatle georgethebeatle added this to the release-candidate milestone Jan 24, 2025
@georgethebeatle georgethebeatle moved this from 🇪🇺 To do to 🔄 In progress in Korifi - Backlog Jan 27, 2025
@georgethebeatle georgethebeatle self-assigned this Jan 27, 2025
georgethebeatle pushed a commit that referenced this issue Jan 30, 2025
* Allow `GAUGE` envelope type for the `envelope_type` query parameters
* The implementation assumes `LOGS` envelope type if not specified. This
  is good enough for the current implementation and we do not have to
  merge logs and gauges. We could change that in future if needed
* When the envelope type is `GAUGE` we ignore all the other query
  paramters as we do not return historical data, i.e. all the gauge
  values represent the current state.

Co-authored-by: Georgi Sabev <[email protected]>
Co-authored-by: Yusmen Zabanov <[email protected]>
Co-authored-by: Danail Branekov <[email protected]>

fixes #3752
danail-branekov added a commit that referenced this issue Jan 30, 2025
* Allow `GAUGE` envelope type for the `envelope_type` query parameters
* The implementation assumes `LOGS` envelope type if not specified. This
  is good enough for the current implementation and we do not have to
  merge logs and gauges. We could change that in future if needed
* When the envelope type is `GAUGE` we ignore all the other query
  paramters as we do not return historical data, i.e. all the gauge
  values represent the current state.

Co-authored-by: Georgi Sabev <[email protected]>
Co-authored-by: Yusmen Zabanov <[email protected]>
Co-authored-by: Danail Branekov <[email protected]>

fixes #3752

Co-authored-by: Georgi Sabev <[email protected]>
Co-authored-by: Yusmen Zabanov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🔄 In progress
Development

No branches or pull requests

1 participant