-
Notifications
You must be signed in to change notification settings - Fork 175
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
10 changed files
with
337 additions
and
162 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,10 +1,10 @@ | ||
| # Tuning | ||
|
|
||
| To get the best performance and latency {{ productName }} implements performance tuning. | ||
| To get the best performance and latency {{productName}} implements performance tuning. | ||
|
|
||
| Performance tuning is enabled by default *when you create a corresponding [NodeConfig](../resources/nodeconfigs.md) for your nodes. | ||
|
|
||
| Because some of the operations it needs to perform are not multitenant or priviledged, the tuning scripts are run in a dedicated system namespace called `scylla-operator-node-tuning`. | ||
| This namespace is created and entirely managed by {{ productName }} and only administrators can access it. | ||
| This namespace is created and entirely managed by {{productName}} and only administrators can access it. | ||
|
|
||
| When a ScyllaCluster Pod is created (and performance tuning is enabled), the Pod initializes but waits until {{ productName }} runs an on-demand Job that will configure the host and the ScyllaDB process accordingly. Only after that it will actually start running ScyllaDB. | ||
| When a ScyllaCluster Pod is created (and performance tuning is enabled), the Pod initializes but waits until {{productName}} runs an on-demand Job that will configure the host and the ScyllaDB process accordingly. Only after that it will actually start running ScyllaDB. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,44 +1,203 @@ | ||
| # GitOps (kubectl) | ||
|
|
||
| ## Manifests location | ||
| sidecar image | ||
| ## Disclaimer | ||
|
|
||
| manifest location | ||
|
|
||
| (move to dedicated section) | ||
| For the ease of use all the following commands reference manifests that come from the same repository as the source code is being built from. | ||
| This means we can't have a pinned reference to the latest z-stream as that is a [chicken-egg problem](https://en.wikipedia.org/wiki/Chicken_or_the_egg). Therefore, we use a rolling tag for the particular branch in our manifests. | ||
| :::{caution} | ||
| For production deployment, you should always replace the {{productName}} image with a stable reference. | ||
| We'd encourage you to use a sha reference, although using full-version tags is also ok. | ||
| ::: | ||
|
|
||
|
|
||
| ## Installation | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| Scylla Operator has a Cert Manager dependency, you have to install it first. | ||
| It can be installed via following command executed in the root directory of repository: | ||
| ```shell | ||
| kubectl apply -f https://{{ repositoryURL }}/blob/{{ revision }}/examples/common/cert-manager.yaml | ||
| ``` | ||
| ```{code-block} bash | ||
| Scylla Operator has a few dependencies that you need to install to your cluster first. | ||
|
|
||
| In case you already have a supported version of each of these dependencies installed in your cluster, you can skip this part. | ||
|
|
||
| #### Cert Manager | ||
|
|
||
| :::{code-block} shell | ||
| :substitutions: | ||
|
|
||
| kubectl apply -f https://{{ repositoryURL }}/blob/{{ revision }}/examples/common/cert-manager.yaml | ||
| ``` | ||
| :::{code-block} | ||
| # Deploy cert-manager. | ||
| kubectl -n=cert-manager apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/third-party/cert-manager.yaml | ||
| ::: | ||
|
|
||
| :::{code-block} shell | ||
| # Wait for CRDs to propagate to all apiservers. | ||
| kubectl wait --for condition=established --timeout=60s crd/certificates.cert-manager.io crd/issuers.cert-manager.io | ||
|
|
||
| # Wait for components that others steps depend on. | ||
| for deploy in cert-manager{,-cainjector,-webhook}; do | ||
| kubectl -n=cert-manager rollout status --timeout=10m deployment.apps/"${deploy}" | ||
| done | ||
|
|
||
| # Wait for webhook CA secret to be created. | ||
| for i in {1..30}; do | ||
| { kubectl -n=cert-manager get secret/cert-manager-webhook-ca && break; } || sleep 1 | ||
| done | ||
| ::: | ||
|
|
||
| #### Prometheus Operator | ||
|
|
||
| :::{code-block} shell | ||
| :substitutions: | ||
|
|
||
| {{repositoryURL}} | ||
| {{ revision }} | ||
| kubectl -n=prometheus-operator apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/third-party/prometheus-operator.yaml | ||
| ::: | ||
| :::{substitution-code} | ||
|
|
||
| {{repositoryURL}} | ||
| {{ revision }} | ||
| :::{code-block} shell | ||
| # Wait for CRDs to propagate to all apiservers. | ||
| kubectl wait --for='condition=established' crd/prometheuses.monitoring.coreos.com crd/prometheusrules.monitoring.coreos.com crd/servicemonitors.monitoring.coreos.com | ||
|
|
||
| # Wait for prometheus operator deployment. | ||
| kubectl -n=prometheus-operator rollout status --timeout=10m deployment.apps/prometheus-operator | ||
|
|
||
| # Wait for webhook CA secret to be created. | ||
| for i in {1..30}; do | ||
| { kubectl -n=cert-manager get secret/cert-manager-webhook-ca && break; } || sleep 1 | ||
| done | ||
| ::: | ||
|
|
||
| ### CustomResourceDefinitions | ||
| ```console | ||
| ### {{productName}} | ||
|
|
||
| ``` | ||
| :::{code-block} shell | ||
| :substitutions: | ||
|
|
||
| ### Operator | ||
| kubectl -n=scylla-operator apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/deploy/operator/operator.yaml | ||
| ::: | ||
|
|
||
| ::::{caution} | ||
| {{productName}} deployment references its own image that it later runs alongside each ScyllaDB instance. Therefore, you have to also replace the image in the environment variable called `SCYLLA_OPERATOR_IMAGE`: | ||
| :::{code-block} yaml | ||
| :linenos: | ||
| :emphasize-lines: 16,19,20 | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: scylla-operator | ||
| namespace: scylla-operator | ||
| # ... | ||
| spec: | ||
| # ... | ||
| template: | ||
| # ... | ||
| spec: | ||
| # ... | ||
| containers: | ||
| - name: scylla-operator | ||
| # ... | ||
| image: docker.io/scylladb/scylla-operator:1.14.0@sha256:8c75c5780e2283f0a8f9734925352716f37e0e7f41007e50ce9b1d9924046fa1 | ||
| env: | ||
| # ... | ||
| - name: SCYLLA_OPERATOR_IMAGE | ||
| value: docker.io/scylladb/scylla-operator:1.14.0@sha256:8c75c5780e2283f0a8f9734925352716f37e0e7f41007e50ce9b1d9924046fa1 | ||
| ::: | ||
| The {{productName}} image value and the `SCYLLA_OPERATOR_IMAGE` shall always match. | ||
| Be careful not to use a rolling tag for any of them to avoid an accidental skew! | ||
| :::: | ||
|
|
||
| :::{code-block} shell | ||
| # Wait for CRDs to propagate to all apiservers. | ||
| kubectl wait --for='condition=established' crd/scyllaclusters.scylla.scylladb.com crd/nodeconfigs.scylla.scylladb.com crd/scyllaoperatorconfigs.scylla.scylladb.com crd/scylladbmonitorings.scylla.scylladb.com | ||
|
|
||
| # Wait for the components to deploy. | ||
| kubectl -n=scylla-operator rollout status --timeout=10m deployment.apps/{scylla-operator,webhook-server} | ||
|
|
||
| # Wait for webhook CA secret to be created. | ||
| for i in {1..30}; do | ||
| { kubectl -n=cert-manager get secret/cert-manager-webhook-ca && break; } || sleep 1 | ||
| done | ||
| ::: | ||
|
|
||
| ### Setting up local storage on nodes and enabling tuning | ||
|
|
||
| :::{caution} | ||
| The following step heavily depends on the platform that you use, the machine type, or the options chosen when creating a node pool. | ||
|
|
||
| Please review the [NodeConfig](../resources/nodeconfigs.md) and adjust it for your platform! | ||
| ::: | ||
|
|
||
| :::::{tab-set} | ||
|
|
||
| ::::{tab-item} GKE (NVMe) | ||
| :::{code-block} shell | ||
| :substitutions: | ||
| kubectl -n=scylla-operator apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/gke/nodeconfig-alpha.yaml | ||
| ::: | ||
| :::: | ||
|
|
||
| ::::{tab-item} EKS (NVMe) | ||
| :::{code-block} shell | ||
| :substitutions: | ||
| kubectl -n=scylla-operator apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/eks/nodeconfig-alpha.yaml | ||
| ::: | ||
| :::: | ||
|
|
||
| ::::{tab-item} Any platform (Loop devices) | ||
| :::{caution} | ||
| This NodeConfig sets up loop devices instead of NVMe disks and is only intended for development purposes when you don't have the NVMe disks available. | ||
| Do not expect meaningful performance with this setup. | ||
| ::: | ||
| :::{code-block} shell | ||
| :substitutions: | ||
| kubectl -n=scylla-operator apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/generic/nodeconfig-alpha.yaml | ||
| ::: | ||
| :::: | ||
|
|
||
| ::::: | ||
|
|
||
| :::{note} | ||
| Performance tuning is enabled for all nodes that are selected by the [NodeConfig](../resources/nodeconfigs.md) by default, unless opted-out. | ||
| ::: | ||
|
|
||
| :::{code-block} shell | ||
| # Wait for the NodeConfig to apply changes to the Kubernetes nodes. | ||
| kubectl wait --for='condition=Reconciled' --timeout=10m nodeconfigs.scylla.scylladb.com/cluster | ||
| ::: | ||
|
|
||
| ### Local CSI driver | ||
|
|
||
| :::{code-block} shell | ||
| :substitutions: | ||
|
|
||
| kubectl -n=local-csi-driver apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/examples/common/local-volume-provisioner/local-csi-driver/{00_namespace,00_scylladb-local-xfs.storageclass,10_csidriver,10_driver.serviceaccount,10_provisioner_clusterrole,20_provisioner_clusterrolebinding,50_daemonset}.yaml | ||
| ::: | ||
|
|
||
| :::{code-block} shell | ||
| # Wait for it to deploy. | ||
| kubectl -n=local-csi-driver rollout status --timeout=10m daemonset.apps/local-csi-driver | ||
| ::: | ||
|
|
||
| ### ScyllaDBManager | ||
|
|
||
| :::::{tab-set} | ||
|
|
||
| ::::{tab-item} Production (sized) | ||
| :::{code-block} shell | ||
| :substitutions: | ||
| kubectl -n=scylla-manager apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/deploy/manager-prod.yaml | ||
| ::: | ||
| :::: | ||
|
|
||
| ::::{tab-item} Development (sized) | ||
| :::{code-block} shell | ||
| :substitutions: | ||
| kubectl -n=scylla-manager apply --server-side -f=https://raw.githubusercontent.com/{{repository}}/{{revision}}/deploy/manager-dev.yaml | ||
| ::: | ||
| :::: | ||
|
|
||
| ::::: | ||
|
|
||
| :::{code-block} shell | ||
| # Wait for it to deploy. | ||
| kubectl -n=local-csi-driver rollout status --timeout=10m daemonset.apps/local-csi-driver | ||
| ::: | ||
|
|
||
| ## Next steps | ||
|
|
||
| Now that you've successfully installed {{productName}}, it's time to look at [how to run ScyllaDB](../resources/scyllaclusters/overview.md). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.