Skip to content

Commit

Permalink
HAP Doc Reorganization & Validation Constraints
Browse files Browse the repository at this point in the history
  • Loading branch information
@ralikio
ralikio committed Jan 16, 2025
1 parent c0ae118 commit 97e26c7
Showing 1 changed file with 18 additions and 18 deletions.
36 changes: 18 additions & 18 deletions docs/contributor/03-10-hyperscaler-account-pool.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@

To provision clusters through Gardener using Runtime Provisioner, Kyma Environment Broker (KEB) requires a hyperscaler (GCP, Azure, AWS, etc.) account/subscription. Managing the available hyperscaler accounts is not in the scope of KEB. Instead, the available accounts are handled by Hyperscaler Account Pool (HAP).

HAP stores credentials for the hyperscaler accounts that have been set up in advance in Kubernetes Secrets. The credentials are stored separately for each provider and tenant. The content of the credentials Secrets may vary for different use cases. The Secrets are labeled with the **hyperscaler-type** and **tenant-name** labels to manage pools of credentials for use by the provisioning process. This way, the in-use credentials and unassigned credentials available for use are tracked. Only the **hyperscaler-type** label is added during Secret creation, and the **tenant-name** label is added when the account respective for a given Secret is claimed. The content of the Secrets is opaque to HAP.
HAP stores credentials for the hyperscaler accounts that have been set up in advance in Kubernetes Secrets. The credentials are stored separately for each provider and tenant. The content of the credentials Secrets may vary for different use cases. The Secrets are labeled with the **hyperscaler-type**, **euAccess**, **shared** and **tenant-name** labels to manage pools of credentials for use by the provisioning process. The **hyperscaler-type** contains hyperscaler name and region information in the format `hyperscaler_type: <HYPERSCALER_NAME>[_<PLATFORM_REGION>][_<HYPERSCALER_REGION>]`, where both `_<PLATFORM_REGION>` and `_<HYPERSCALER_REGION>` are optional. The **euAccess** and **shared** labels contain boolean values and they used to divide existing pools to secrets used by EU restricted regions and secrets shared by multiple Global Accounts. The **tenant-name** label is added when the account respective for a given Secret is claimed and it is the only one not added during Secret creation. This way, the in-use credentials and unassigned credentials available for use are tracked. The content of the Secrets is opaque to HAP.

The Secrets are stored in a Gardener seed cluster pointed to by HAP. They are available within a given Gardener project specified in the KEB and Runtime Provisioner configuration. This configuration uses a `kubeconfig` that gives KEB and Runtime Provisioner access to a specific Gardener seed cluster, which, in turn, enables access to those Secrets.

This diagram shows the HAP workflow:

![hap-workflow](../assets/hap-flow.drawio.svg)

Before a new cluster is provisioned, KEB queries for a Secret based on the **tenant-name** and **hyperscaler-type** labels.
Before a new cluster is provisioned, KEB queries for a Secret based on the mandatory **hyperscaler-type** and optional **tenant-name**, **euAccess** and **shared** labels.

If a Secret is found, KEB uses the credentials stored in this Secret. If a matching Secret is not found, KEB queries again for an unassigned Secret for a given hyperscaler and adds the **tenant-name** label to claim the account and use the credentials for provisioning.

One tenant can use only one account per given hyperscaler type.
Expand Down Expand Up @@ -84,25 +85,18 @@ metadata:
hyperscaler-type: "gcp_cf-sa30"
```

## HAP Configuration

<!-- selection algorithm -->
When selecting a Secret for a given hyperscaler, KEB evaluates a set of rules to determine what secret bindings it should query for a given plan and region that SKR is provisioned in. The search looks for a secret bindings with `hyperscaler-type` label only (always used in the search) or additionally with one of or both `euAccess: true` and `shared: true` labels.

`hyperscaler-type` label is the one that does contain region information in the format `hyperscaler_type: <HYPERSCALER_NAME>[_<PLATFORM_REGION>][_<CLUSTER_REGION>]`, where both `_<PLATFORM_REGION>` and `_<CLUSTER_REGION>` are optional.

The configuration is done by specifying a list in KEB helm values. There are four possible properties, with rules evaluated for given cluster, that can be set:
* `hap.platformRegionRules` - if evaluated to true the `_<PLATFOR_REGION>` is appended to the `hyperscaler-type` label when searching, refered to as platform region based search,
* `hap.clusterRegionRules` - if evaluated to true the `_<CLUSTER_REGION>` is appended to the `hyperscaler-type` label when searching, refered to as cluster region based search,
* `hap.sharedRules` - if evaluated to true the `shared: true` label is used when searching, refered to as shared based search,
* `hap.euAccessRules` - if evaluated to true the `shared: true` label is used when searching, refered to as euAccess based search.
## Secret Bindings Selection Rules

![alt text](image.png)
<!-- rules Overview -->
HAP evaluates a set of rules to determine what labels to use when querying secret bindings. Input to the rules consists of an SKR's plan, hyperscaler and region. If evaluated to true the rules modify or add labels used in the secret bindings resource query. There are four possible rules to configure:
* `hap.platformRegionRule` - if evaluated to true the `_<PLATFOR_REGION>` is appended to the `hyperscaler-type` label when searching, refered to as platform region based search,
* `hap.clusterRegionRule` - if evaluated to true the `_<HYPERSCALER_REGION>` is appended to the `hyperscaler-type` label when searching, refered to as cluster region based search,
* `hap.sharedRule` - if evaluated to true the `shared: true` label is used when searching, refered to as shared based search,
* `hap.euAccessRule` - if evaluated to true the `shared: true` label is used when searching, refered to as euAccess based search.
The configuration is done by specifying above rules in KEB helm values.

<!-- rules format -->
Each configuration consists of a list (The list entries are separated with semicolons) of plans that the rule applies to. Additionally, a plan can be extended with `:region` suffix that makes the rule evaluate to true only if a cluster is provisioned in the specified region.

The property accepts a list of key value in the format `<PLAN_ID_1>:<REGION_ID_1>;<PLAN_ID_2>:<REGION_ID_2>`. Either plan or region (but never both) can be specified as wildcard `*` meaning all plans or regions should apply for specific second value. The following example lists valid and invalid configuration values:
Each rule consists of a semicolon separated list of plans that the rule applies to. Additionally, a plan can be extended with `:region` suffix that makes the rule evaluate to true only if a cluster is provisioned in the specified region. List entries comply with the format `<PLAN_ID_1>:<REGION_ID_1>;<PLAN_ID_2>:<REGION_ID_2>`. Either plan or region (but never both) can be specified as wildcard `*` meaning all plans or regions should apply for specific second value. The following example lists valid and invalid configuration values:
* `trial` - valid, rule will be evaluated to true for the `trial` plan in all regions,
* `trial:*` - valid, rule will be evaluated to true for the `trial` plan in all regions,
* `trial:eu` - valid, rule will be evaluated to true for the `trial` plan in the `eu` region,
Expand All @@ -114,4 +108,10 @@ The property accepts a list of key value in the format `<PLAN_ID_1>:<REGION_ID_1
* `*:eu;trial:eu` - valid, rule will be evaluated to true for all plans in the `eu` region and for the `trial` plan in the `eu` region, configuration can be duplicated
* `trial:eu;trial:gcp` - valid, rule will be evaluated to true for trials plans but only in `eu` and `gcp` regions,

<!-- search construction -->
![alt text](image.png)

<!-- TODO: rules validation -->
Rules validation is done during the KEB startup. If the configuration is invalid, KEB will not start and an error message will be displayed in the logs. The constraints used for validation include:
* Rules format check - all the rules must comply with the format specified above.
* Plan ?and region? existence check.

0 comments on commit 97e26c7

Please sign in to comment.