Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job runner details #124

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+45 −4
Open

Job runner details #124

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/runner-type/runner-type.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Runner Types

Metrics are labelled with the runner type. The logic for determining the runner type is as follows -

If the runner tag is in the [github_hosted_runner_labels](./gh_actions_exporter/config.py) list the runner type will be `github-hosted` else the runner type will be `self-hosted`. Additional `github-hosted` runner tags can be added via config.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how the doc was rendered (through mkdocs or github), through github at least couldn't get it to redirect unless I go up a couple of folders.

Also added some note for my future self as I was wondering how we could handle GitHub hosted larger runners with this setting.

Suggested change
If the runner tag is in the [github_hosted_runner_labels](./gh_actions_exporter/config.py) list the runner type will be `github-hosted` else the runner type will be `self-hosted`. Additional `github-hosted` runner tags can be added via config.
- If the runner tag is in the [github_hosted_runner_labels](../../gh_actions_exporter/config.py)
list, the runner type will be `github-hosted` else the runner type will be `self-hosted`.
- Additional `github-hosted` runner tags can be added via the config file,
for example GitHub hosted larger runners as their name is defined
by the owner of the GitHub organization.

14 changes: 14 additions & 0 deletions gh_actions_exporter/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,20 @@ class Settings(BaseSettings):
github_app_id: Optional[int]
github_app_installation_id: Optional[int]
github_app_private_key: Optional[SecretStr]
github_hosted_runner_labels: Optional[List[str]] = [
"ubuntu-latest",
"ubuntu-24.04",
"ubuntu-22.04",
"ubuntu-20.04",
"windows-latest",
"windows-2022",
"windows-2019",
"macos-latest",
"macos-14",
"macos-13",
"macos-12",
"macos-11"
]

class Config:
config: ConfigFile = ConfigFile()
Expand Down
13 changes: 9 additions & 4 deletions gh_actions_exporter/metrics.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ def __init__(self, settings: Settings):
self.job_labelnames = self.common_labelnames.copy() + [
"job_name",
"runner_type",
"runner_labels",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of query or filter were you trying to do when deciding to add the runner_labels and have you tried using the relabel option?

I see no issue in adding it, just wanna make sure you tried all the options available to you.

For example in our org, our self-hosted runners are defined with multiple labels:

  • The os (ubuntu-20.04)
  • The flavor (large)
  • And in some case the cloud provider (aws, azure, etc)

And we use the relabel feature to retrieve those value and create new label like:

  • cloud=aws
  • flavor=large

Which makes it easier for us to query around.

Copy link
Author

@apj0nes apj0nes Jun 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does look promising. Would you have an example yaml config for achieving the example above?

Copy link
Contributor

@tcarmet tcarmet Jun 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do yes, this is an example of what we've been playing with:

job_relabelling:
  - label: cloud
    default: github-hosted
    values:
      - azure
      - aws
    type: name # retrieve the info from the runner name (defaults to labels).
  - label: image
    default: unknown
    values:
      - ubuntu-latest
      - ubuntu-20.04
      - ubuntu-22.04
      - rocky-8
      - redhat-8
      - redhat-9
  - label: flavor
    default: medium # consider github-hosted flavors as medium
    values:
      - small
      - medium
      - large
      - xlarge

And sorry for the lack of documentation on those 😅 , I'll take note on my end and make an effort towards that in the coming days.

]
for relabel in self.settings.job_relabelling:
self.job_labelnames.append(relabel.label)
Expand Down Expand Up @@ -148,11 +149,14 @@ def workflow_labels(self, webhook: WebHook) -> dict:
branch=branch,
event=webhook.workflow_run.event,
)

def runner_type(self, webhook: WebHook) -> str:
if "self-hosted" in webhook.workflow_job.labels:
return "self-hosted"
return "github-hosted"
if set(webhook.workflow_job.labels) <= set(self.settings.github_hosted_runner_labels):
return "github-hosted"
return "self-hosted"

def runner_labels(self, webhook: WebHook) -> str:
return ','.join(webhook.workflow_job.labels)

def relabel_job_labels(
self, relabel: Relabel, labels: List[str]
Expand Down Expand Up @@ -180,6 +184,7 @@ def job_labels(self, webhook: WebHook, settings: Settings) -> dict:
repository_visibility=webhook.repository.visibility,
repository=webhook.repository.full_name,
workflow_name=webhook.workflow_job.workflow_name,
runner_labels=self.runner_labels(webhook),
)

for relabel in settings.job_relabelling:
Expand Down
17 changes: 17 additions & 0 deletions tests/api/metrics/test_job.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,20 @@ def test_skipped_job(override_job_config, client, workflow_job, headers):
workflow_job["workflow_job"]["completed_at"] = "2021-11-29T14:59:57Z"
response = client.post("/webhook", json=workflow_job, headers=headers)
assert response.status_code == 202

@pytest.mark.parametrize("labels,expected_runner_type", [
(["warp-ubuntu-latest-x64-2x"], "self-hosted"),
(["self-hosted","linux"], "self-hosted"),
(["ubuntu-latest"], "github-hosted")
])
def test_runner_type_and_labels(client, workflow_job, headers, labels, expected_runner_type):
workflow_job["workflow_job"]["labels"] = labels
response = client.post("/webhook", json=workflow_job, headers=headers)
assert response.status_code == 202

metrics = client.get("/metrics")
assert metrics.status_code == 200

for line in metrics.text.split("\n"):
if "github_actions_job_start_duration_seconds_bucket{" in line:
assert "runner_labels=\"%s\",runner_type=\"%s\"" % (','.join(labels), expected_runner_type) in line
Loading