Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Grafana Cloud Fleet Management #1989

Merged
merged 30 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ed69068
Add Fleet Management fields to Cloud Stack
johannaojeling Dec 12, 2024
76b2773
Create Fleet Management client
johannaojeling Dec 12, 2024
91b5104
Configure Fleet Management client in provider
johannaojeling Dec 12, 2024
24189bb
Add Fleet Management category
johannaojeling Dec 12, 2024
6705195
Implement utility functions
johannaojeling Dec 12, 2024
e723aa0
Implement common resource function
johannaojeling Dec 12, 2024
826e746
Create custom type for Alloy config
johannaojeling Dec 12, 2024
91573f5
Create Collector model
johannaojeling Dec 12, 2024
3bba775
Create Collector resource
johannaojeling Dec 12, 2024
1163caa
Create Pipeline model
johannaojeling Dec 12, 2024
4b49f52
Create Pipeline resource
johannaojeling Dec 12, 2024
36fd254
Include resources in provider
johannaojeling Dec 12, 2024
48b8e92
Add resource examples
johannaojeling Dec 16, 2024
7c393d9
Update provider examples
johannaojeling Dec 19, 2024
0d07828
Include Fleet Management in TestAccExamples
johannaojeling Dec 16, 2024
9959cf1
Add acceptance tests
johannaojeling Dec 18, 2024
dcc73be
Generate documentation
johannaojeling Dec 16, 2024
01baad9
Add Fleet Management code owners
johannaojeling Jan 10, 2025
37aefc3
Add resource descriptions with public preview note
johannaojeling Jan 23, 2025
93c1d9b
Clarify experimental status
johannaojeling Jan 29, 2025
724de0f
Create custom Prometheus matcher type
johannaojeling Jan 28, 2025
43db807
Refactor custom Alloy config type
johannaojeling Jan 28, 2025
d15aeb9
Create custom generic list type
johannaojeling Jan 28, 2025
d368544
Use custom types and TF type conversion in resources
johannaojeling Jan 28, 2025
b8f52ef
Use stack data source in provider example
johannaojeling Jan 29, 2025
22edc4c
Validate fleet_management_auth attribute
johannaojeling Jan 29, 2025
ca38135
Rename attribute_overrides to remote_attributes
johannaojeling Jan 29, 2025
ca8e204
Validate fleet_management_url attribute
johannaojeling Jan 29, 2025
2a82843
Recreate resources if deleted externally
johannaojeling Jan 30, 2025
cfb92eb
Use latest Fleet Management API with RemoteAttributes
johannaojeling Feb 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
/internal/resources/cloud/* @grafana/platform-monitoring @grafana/grafana-com-maintainers
/internal/resources/cloudprovider/* @grafana/platform-monitoring @grafana/middleware-apps
/internal/resources/connections/* @grafana/platform-monitoring @grafana/middleware-apps
/internal/resources/fleetmanagement/* @grafana/platform-monitoring @grafana/fleet-management-backend
/internal/resources/machinelearning/* @grafana/platform-monitoring @grafana/machine-learning
/internal/resources/oncall/* @grafana/platform-monitoring @grafana/grafana-irm-backend
/internal/resources/slo/* @grafana/platform-monitoring @grafana/slo-squad
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/acc-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ jobs:
GRAFANA_CLOUD_PROVIDER_ACCESS_TOKEN=cloudprovider-tests:access-token
GRAFANA_CLOUD_PROVIDER_AWS_ROLE_ARN=cloudprovider-tests:aws-role-arn
GRAFANA_CLOUD_PROVIDER_TEST_STACK_ID=cloudprovider-tests:test-stack-id
GRAFANA_FLEET_MANAGEMENT_AUTH=cloud-instance-tests:fleet-management-auth
GRAFANA_FLEET_MANAGEMENT_URL=cloud-instance-tests:fleet-management-url
- uses: iFaxity/wait-on-action@a7d13170ec542bdca4ef8ac4b15e9c6aa00a6866 # v1.2.1
with:
resource: ${{ env.GRAFANA_URL }}
Expand Down
4 changes: 4 additions & 0 deletions docs/data-sources/cloud_stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ available at “https://<stack_slug>.grafana.net".
- `alertmanager_user_id` (Number) User ID of the Alertmanager instance configured for this stack.
- `cluster_slug` (String) Slug of the cluster where this stack resides.
- `description` (String) Description of stack.
- `fleet_management_name` (String) Name of the Fleet Management instance configured for this stack.
- `fleet_management_status` (String) Status of the Fleet Management instance configured for this stack.
- `fleet_management_url` (String) Base URL of the Fleet Management instance configured for this stack.
- `fleet_management_user_id` (Number) User ID of the Fleet Management instance configured for this stack.
- `graphite_name` (String)
- `graphite_status` (String)
- `graphite_url` (String)
Expand Down
94 changes: 94 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,8 @@ resource "grafana_oncall_escalation" "example_notify_step" {
- `cloud_provider_url` (String) A Grafana Cloud Provider backend address. May alternatively be set via the `GRAFANA_CLOUD_PROVIDER_URL` environment variable.
- `connections_api_access_token` (String, Sensitive) A Grafana Connections API access token. May alternatively be set via the `GRAFANA_CONNECTIONS_API_ACCESS_TOKEN` environment variable.
- `connections_api_url` (String) A Grafana Connections API address. May alternatively be set via the `GRAFANA_CONNECTIONS_API_URL` environment variable.
- `fleet_management_auth` (String, Sensitive) A Grafana Fleet Management basic auth in the `username:password` format. May alternatively be set via the `GRAFANA_FLEET_MANAGEMENT_AUTH` environment variable.
- `fleet_management_url` (String) A Grafana Fleet Management API address. May alternatively be set via the `GRAFANA_FLEET_MANAGEMENT_URL` environment variable.
- `http_headers` (Map of String, Sensitive) Optional. HTTP headers mapping keys to values used for accessing the Grafana and Grafana Cloud APIs. May alternatively be set via the `GRAFANA_HTTP_HEADERS` environment variable in JSON format.
- `insecure_skip_verify` (Boolean) Skip TLS certificate verification. May alternatively be set via the `GRAFANA_INSECURE_SKIP_VERIFY` environment variable.
- `oncall_access_token` (String, Sensitive) A Grafana OnCall access token. May alternatively be set via the `GRAFANA_ONCALL_ACCESS_TOKEN` environment variable.
Expand Down Expand Up @@ -450,6 +452,91 @@ provider "grafana" {
}
```

### Managing Grafana Fleet Management

```terraform
// Variables
variable "cloud_access_policy_token" {
type = string
description = "Cloud access policy token with scopes: accesspolicies:read|write|delete, stacks:read"
}

variable "stack_slug" {
type = string
description = "Subdomain that the Grafana Cloud instance is available at: https://<stack_slug>.grafana.net"
}

// Step 1: Retrieve stack details
provider "grafana" {
alias = "cloud"

cloud_access_policy_token = var.cloud_access_policy_token
}

data "grafana_cloud_stack" "stack" {
provider = grafana.cloud

slug = var.stack_slug
}

// Step 2: Create an access policy and token for Fleet Management
resource "grafana_cloud_access_policy" "policy" {
provider = grafana.cloud

name = "fleet-management-policy"
region = data.grafana_cloud_stack.stack.region_slug

scopes = [
"fleet-management:read",
"fleet-management:write"
]

realm {
type = "stack"
identifier = data.grafana_cloud_stack.stack.id
}
}

resource "grafana_cloud_access_policy_token" "token" {
provider = grafana.cloud

name = "fleet-management-token"
region = grafana_cloud_access_policy.policy.region
access_policy_id = grafana_cloud_access_policy.policy.policy_id
}

// Step 3: Interact with Fleet Management
provider "grafana" {
alias = "fm"

fleet_management_auth = "${data.grafana_cloud_stack.stack.fleet_management_user_id}:${grafana_cloud_access_policy_token.token.token}"
fleet_management_url = data.grafana_cloud_stack.stack.fleet_management_url
}

resource "grafana_fleet_management_collector" "collector" {
provider = grafana.fm

id = "my_collector"
remote_attributes = {
"env" = "PROD",
"owner" = "TEAM-A"
}
enabled = true
}

resource "grafana_fleet_management_pipeline" "pipeline" {
provider = grafana.fm

name = "my_pipeline"
contents = file("config.alloy")
matchers = [
"collector.os=\"linux\"",
"env=\"PROD\""
]
enabled = true
}
```

## Authentication

One, or many, of the following authentication settings must be set. Each authentication setting allows a subset of resources to be used
Expand Down Expand Up @@ -483,3 +570,10 @@ To create one, follow the instructions in the [obtaining cloud provider access t
An access policy token created on the [Grafana Cloud Portal](https://grafana.com/docs/grafana-cloud/account-management/authentication-and-permissions/access-policies/using-an-access-policy-token/) to manage
connections resources, such as Metrics Endpoint jobs.
For guidance on creating one, see section [obtaining connections access token](#obtaining-connections-access-token).

### `fleet_management_auth`

[Grafana Fleet Management](https://grafana.com/docs/grafana-cloud/send-data/fleet-management/api-reference/)
uses basic auth to allow access to the API, where the username is the Fleet Management instance ID and the
password is the API token. You can access the instance ID and request a new Fleet Management API token on the
Connections -> Collector -> Fleet Management page, in the API tab.
4 changes: 4 additions & 0 deletions docs/resources/cloud_stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ resource "grafana_cloud_stack" "test" {
- `alertmanager_url` (String) Base URL of the Alertmanager instance configured for this stack.
- `alertmanager_user_id` (Number) User ID of the Alertmanager instance configured for this stack.
- `cluster_slug` (String) Slug of the cluster where this stack resides.
- `fleet_management_name` (String) Name of the Fleet Management instance configured for this stack.
- `fleet_management_status` (String) Status of the Fleet Management instance configured for this stack.
- `fleet_management_url` (String) Base URL of the Fleet Management instance configured for this stack.
- `fleet_management_user_id` (Number) User ID of the Fleet Management instance configured for this stack.
- `graphite_name` (String)
- `graphite_status` (String)
- `graphite_url` (String)
Expand Down
58 changes: 58 additions & 0 deletions docs/resources/fleet_management_collector.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "grafana_fleet_management_collector Resource - terraform-provider-grafana"
subcategory: "Fleet Management"
description: |-
Manages Grafana Fleet Management collectors.
Official documentation https://grafana.com/docs/grafana-cloud/send-data/fleet-management/API documentation https://grafana.com/docs/grafana-cloud/send-data/fleet-management/api-reference/collector-api/
Note: Fleet Management is in public preview https://grafana.com/docs/release-life-cycle/#public-preview and this resource is experimental. Grafana Labs offers limited support, and breaking changes might occur.
Required access policy scopes:
fleet-management:readfleet-management:write
---

# grafana_fleet_management_collector (Resource)

Manages Grafana Fleet Management collectors.

* [Official documentation](https://grafana.com/docs/grafana-cloud/send-data/fleet-management/)
* [API documentation](https://grafana.com/docs/grafana-cloud/send-data/fleet-management/api-reference/collector-api/)

**Note:** Fleet Management is in [public preview](https://grafana.com/docs/release-life-cycle/#public-preview) and this resource is experimental. Grafana Labs offers limited support, and breaking changes might occur.

Required access policy scopes:

* fleet-management:read
* fleet-management:write

## Example Usage

```terraform
resource "grafana_fleet_management_collector" "test" {
id = "my_collector"
remote_attributes = {
"env" = "PROD",
"owner" = "TEAM-A"
}
enabled = true
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `id` (String) ID of the collector

### Optional

- `enabled` (Boolean) Whether the collector is enabled or not
- `remote_attributes` (Map of String) Remote attributes for the collector

## Import

Import is supported using the following syntax:

```shell
terraform import grafana_fleet_management_collector.name "{{ id }}"
```
64 changes: 64 additions & 0 deletions docs/resources/fleet_management_pipeline.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
# generated by https://github.com/hashicorp/terraform-plugin-docs
page_title: "grafana_fleet_management_pipeline Resource - terraform-provider-grafana"
subcategory: "Fleet Management"
description: |-
Manages Grafana Fleet Management pipelines.
Official documentation https://grafana.com/docs/grafana-cloud/send-data/fleet-management/API documentation https://grafana.com/docs/grafana-cloud/send-data/fleet-management/api-reference/pipeline-api/
Note: Fleet Management is in public preview https://grafana.com/docs/release-life-cycle/#public-preview and this resource is experimental. Grafana Labs offers limited support, and breaking changes might occur.
Required access policy scopes:
fleet-management:readfleet-management:write
---

# grafana_fleet_management_pipeline (Resource)

Manages Grafana Fleet Management pipelines.

* [Official documentation](https://grafana.com/docs/grafana-cloud/send-data/fleet-management/)
* [API documentation](https://grafana.com/docs/grafana-cloud/send-data/fleet-management/api-reference/pipeline-api/)

**Note:** Fleet Management is in [public preview](https://grafana.com/docs/release-life-cycle/#public-preview) and this resource is experimental. Grafana Labs offers limited support, and breaking changes might occur.

Required access policy scopes:

* fleet-management:read
* fleet-management:write

## Example Usage

```terraform
resource "grafana_fleet_management_pipeline" "test" {
name = "my_pipeline"
contents = file("config.alloy")
matchers = [
"collector.os=~\".*\"",
"env=\"PROD\""
]
enabled = true
}
```

<!-- schema generated by tfplugindocs -->
## Schema

### Required

- `contents` (String) Configuration contents of the pipeline to be used by collectors
- `name` (String) Name of the pipeline which is the unique identifier for the pipeline

### Optional

- `enabled` (Boolean) Whether the pipeline is enabled for collectors
- `matchers` (List of String) Used to match against collectors and assign pipelines to them; follows the syntax of Prometheus Alertmanager matchers

### Read-Only

- `id` (String) Server-assigned ID of the pipeline

## Import

Import is supported using the following syntax:

```shell
terraform import grafana_fleet_management_pipeline.name "{{ name }}"
```
80 changes: 80 additions & 0 deletions examples/provider/provider-fleet-management.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Variables
variable "cloud_access_policy_token" {
type = string
description = "Cloud access policy token with scopes: accesspolicies:read|write|delete, stacks:read"
}

variable "stack_slug" {
type = string
description = "Subdomain that the Grafana Cloud instance is available at: https://<stack_slug>.grafana.net"
}

// Step 1: Retrieve stack details
provider "grafana" {
alias = "cloud"

cloud_access_policy_token = var.cloud_access_policy_token
}

data "grafana_cloud_stack" "stack" {
provider = grafana.cloud

slug = var.stack_slug
}

// Step 2: Create an access policy and token for Fleet Management
resource "grafana_cloud_access_policy" "policy" {
provider = grafana.cloud

name = "fleet-management-policy"
region = data.grafana_cloud_stack.stack.region_slug

scopes = [
"fleet-management:read",
"fleet-management:write"
]

realm {
type = "stack"
identifier = data.grafana_cloud_stack.stack.id
}
}

resource "grafana_cloud_access_policy_token" "token" {
provider = grafana.cloud

name = "fleet-management-token"
region = grafana_cloud_access_policy.policy.region
access_policy_id = grafana_cloud_access_policy.policy.policy_id
}

// Step 3: Interact with Fleet Management
provider "grafana" {
alias = "fm"

fleet_management_auth = "${data.grafana_cloud_stack.stack.fleet_management_user_id}:${grafana_cloud_access_policy_token.token.token}"
fleet_management_url = data.grafana_cloud_stack.stack.fleet_management_url
}

resource "grafana_fleet_management_collector" "collector" {
provider = grafana.fm

id = "my_collector"
remote_attributes = {
"env" = "PROD",
"owner" = "TEAM-A"
}
enabled = true
}

resource "grafana_fleet_management_pipeline" "pipeline" {
provider = grafana.fm

name = "my_pipeline"
contents = file("config.alloy")
matchers = [
"collector.os=\"linux\"",
"env=\"PROD\""
]
enabled = true
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import grafana_fleet_management_collector.name "{{ id }}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
resource "grafana_fleet_management_collector" "test" {
id = "my_collector"
remote_attributes = {
"env" = "PROD",
"owner" = "TEAM-A"
}
enabled = true
}
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
terraform import grafana_fleet_management_pipeline.name "{{ name }}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
resource "grafana_fleet_management_pipeline" "test" {
name = "my_pipeline"
contents = file("config.alloy")
matchers = [
"collector.os=~\".*\"",
"env=\"PROD\""
]
enabled = true
}
Loading
Loading