Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade #407

Merged
merged 5 commits into from
Apr 10, 2024
Merged

Upgrade #407

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6f82353
docs: added new contents in the upgrade document
balaharish7 Apr 8, 2024
498dba5
docs: Created and modified the Upgrade document
balaharish7 Apr 8, 2024
7dded2d
docs: Created and modified the Upgrade document
balaharish7 Apr 8, 2024
ed8813d
docs: added an info panel
balaharish7 Apr 10, 2024
7b3d9b9
docs: modified the doc as per the comments
balaharish7 Apr 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
187 changes: 80 additions & 107 deletions docs/main/user-guides/upgrades.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,156 +5,129 @@ keywords:
- Upgrading OpenEBS
- OpenEBS upgrade
- Supported upgrade paths
description: Upgrade to the latest OpenEBS 2.11.0 version is supported only from v1.0.0 and later.
description: Upgrade to the latest OpenEBS 4.0.0 version.
---

Latest stable version of OpenEBS is 3.10.0. Check the release notes [here](https://github.com/openebs/openebs/releases/tag/v3.10.0).
:::important
Upgrade from OpenEBS 3.x to OpenEBS 4.0.0 is only supported for the below storages installed from OpenEBS 3.x.

Upgrade to the latest cStor or Jiva version is supported only from 1.12.0 or later. The steps for upgrading can be found [here](https://github.com/openebs/upgrade/blob/develop/docs/upgrade.md).
- Local PV Hostpath
- Local PV LVM
- Local PV ZFS
- Replicated Storage (a.k.a Replicated Engine and f.k.a Mayastor)

:::note
- The community e2e pipelines verify upgrade testing only from non-deprecated releases (1.12 and higher) to 3.4.0. If you are running on release older than 1.12, OpenEBS recommends you upgrade to the latest version as soon as possible.
- OpenEBS has deprecated arch specific container images in favor of multi-arch container images. After 2.6, the arch specific images are not pushed to Docker or Quay repositories. For example, images like `cstor-pool-arm64:2.8.0` should be replaced with corresponding multi-arch image `cstor-pool:2.8.0`. For further queries or support, please reach out to [OpenEBS Community](https://kubernetes.slack.com/archives/CUAKPFU78) for helping you with the upgrade.
- If you are upgrading Jiva volumes that are running in version 1.6 and 1.7, you must use these [pre-upgrade steps](https://github.com/openebs/charts/tree/gh-pages/scripts/jiva-tools) to check if your jiva volumes are impacted by [#2956](https://github.com/openebs/openebs/issues/2956). If they are, please reach out to [OpenEBS Community](https://kubernetes.slack.com/archives/CUAKPFU78) for helping you with the upgrade.
See the [migration documentation](../user-guides/data-migration/migration-overview.md) for other storages.
:::

## Mayastor upgrade

Upgrade for Mayastor is supported from v2.0.x to v2.1.0 and above. Before we get into the upgrade documentation for Mayastor, it is important that we understand that there are two ways of deploying Mayastor (since v2.0.0) on to a kubernetes cluster -- the ['mayastor' helm chart](https://github.com/openebs/mayastor-extensions/tree/develop/chart) and the ['openebs' helm chart](https://github.com/openebs/charts/tree/main/charts/openebs). The upgrade instructions differ for these two deployment strategies. The first step of the upgrade is to identify the helm chart that you have used to deploy Mayastor. If you are using the Mayastor with the openebs helm chart, then the instructions for upgrading the mayastor helm chart do not apply to you.
## Overview

### Identifying the helm chart
This upgrade flow would allow users to upgrade to the latest OpenEBS version 4.0.0 which is a unified installer for three Local Storages (a.k.a Local Engines) Local PV HostPath, Local PV LVM, Local PV ZFS, and one Replicated Storage (a.k.a Replicated Engine and f.k.a Mayastor).
As a part of upgrade to OpenEBS 4.0.0, the helm chart would install all four engines irrespective of the engine the user was using prior to the upgrade.

You'll need the helm v3 binary to be on your $PATH. Execute the following command to check if you have helm v3:
```
$ helm version --short
v3.9.4+gdbc6d8e
```
:::info
During the upgrade, advanced users who are only interested in Local PV Storage, can disable the Replicated Storage by setting the below option:
balaharish7 marked this conversation as resolved.
Show resolved Hide resolved

Execute the following command to list the helm releases on the namespace where Mayastor is installed:
```
$ helm list -n openebs
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
openebs-mayastor openebs 1 2023-05-26 22:03:15.473035 +0530 IST deployed openebs-3.6.0 3.6.0
--set mayastor.enabled=false
```
If the value of the 'CHART' column in the row which contains the helm chart for Mayastor, in the output of the above command, reads 'openebs-...', then your Mayastor deployment uses the openebs helm chart. If the CHART column instead reads 'mayastor-...', then it makes use of the mayastor helm chart.
[Click here if your Mayastor deployment uses the 'mayastor' helm chart](https://mayastor.gitbook.io/introduction/additional-information/upgrade).
:::

Continue down this page if your Mayastor deployment uses the 'openebs' helm chart.
## Local Storage

### Upgrading Mayastor installed from the openebs helm chart
This section describes the Local Storage upgrade from OpenEBS chart 3.x to OpenEBS 4.0.0. The upgrade process for Local PV Hostpath, Local PV LVM and Local PV ZFS are largely identical, with a few changes in helm values depending on the Local PV Storage variant we are upgrading from.

#### 1. Planning the upgrade
Supported upgrade paths for Mayastor, installed with the openebs chart, are...
Source versions: 3.4.x or above
Target versions: 3.7.y or above
All source versions may upgrade to any of the target versions, so long as the release semver of the target version is greater than that of the source version.
1. Before upgrading to OpenEBS 4.0.0, the localpv-provisioner deployment needs to be removed to prevent label-selector patch issue while upgrading.

To check for the latest released version of Mayastor available with the openebs helm chart, execute the following commands -- `helm repo update`, `helm search repo openebs --version ">a.b.c"`, where a.b.c is your source version, i.e. the version installed on your cluster. E.g.:
```
$ helm repo update openebs
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "openebs" chart repository
Update Complete. ⎈Happy Helming!⎈

$ helm search repo openebs --version ">3.5.0"
NAME CHART VERSION APP VERSION DESCRIPTION
openebs/openebs 3.7.0 3.7.0 Containerized Attached Storage for Kubernetes
kubectl -n openebs delete deploy -l openebs.io/component-name=openebs-localpv-provisioner --ignore-not-found
```

If you want to list all of the versions available for upgrade, execute the following commands:
```
$ helm repo update openebs
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "openebs" chart repository
Update Complete. ⎈Happy Helming!⎈
2. Execute the 4.0.0 upgrade command.

$ # Assuming source version of 3.5.0
$ helm search repo openebs --version ">3.5.0,>=3.7.0" -l
NAME CHART VERSION APP VERSION DESCRIPTION
openebs/openebs 3.7.0 3.7.0 Containerized Attached Storage for Kubernetes
```
Pick one from the listed version(s).
helm repo update helm upgrade openebs openebs/openebs -n openebs --create-namespace --reuse-values --version 4.0.0
```

#### 2. Building the `helm upgrade` command
The openebs helm chart embeds in to itself all of the OpenEBS storage engines. If you are using more than one of the openebs helm chart's storage engines (e.g. using localpv-provisioner alongside mayastor), then you'd have to include the upgrade options for those storage engines as well into the helm upgrade command.
:::note
If the upgrade is from Local PV LVM or Local PV ZFS storage solution, additional helm values must be specified with the above command to prevent upgrade process conflicts. The installed CRDs in 3.x would be in conflict with the CRDs in 4.0.0 as the chart structure has changed. Hence, they must be disabled.

Going forward, this turtorial assumes that your helm chart release-name is 'openebs-mayastor', that it is installed in the 'openebs' namespace of the kubernetes cluster, it has a source version of 3.6.0 and the target version for the upgrade is 3.7.0.

Execute this shell script/command to get the version of mayastor helm subchart that a particular openebs chart uses:
```
$ # Using 3.7.0 for the input version of the
$ IMG_TAG=$(helm show chart openebs/openebs --version 3.5.0 | grep "name: mayastor" -A 2 | tail -n 1 | tr -d " " | sed -e s/version:/v/)
$ echo $IMG_TAG
v2.2.0
```
The image tag on a released mayastor chart is the value of the chart version, with a 'v' prefix. The above command displays the output after adding the 'v' prefix.
- For Upgrade from Local PV LVM, use

The helm upgrade command with the default options (and using variable from above) should look something like this:
```
helm upgrade openebs openebs/openebs -n openebs --version 3.7.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.7.0 --reuse-values
--set lvm-localpv.crds.lvmLocalPv.enabled=false
```
Refer to the documentation at to see all of the available configuration options:
- https://github.com/openebs/mayastor-extensions/blob/<release-tag\>/chart/README.md
- https://github.com/openebs/mayastor-extensions/blob/<release-tag\>/chart/values.yaml
(e.g.: https://github.com/openebs/mayastor-extensions/blob/v2.2.0/chart/README.md, https://github.com/openebs/mayastor-extensions/blob/v2.2.0/chart/values.yaml)
Prefix any option you'd want to set using the '--set' option, with 'mayastor', like shown in the default command above for the 'image.tag' option. This is due to the mayastor chart being a [subchart of the openebs chart](https://github.com/openebs/charts/blob/openebs-3.7.0/charts/openebs/values.yaml#L400).

Because the mayastor chart is in active development, its helm chart values options may change over time and the convenience of the '--reuse-values' flags may also introduce nil yaml values for values which are required by helm's template engine. The 3.4.0, 3.4.1 and 3.5.0 releases face this issue. These releases require additional options to perform the default upgrade to 3.7.0 or above versions.
Sample helm upgrade command from 3.4.0 to 3.7.0 or above versions:
```
helm upgrade openebs-mayastor openebs/openebs -n openebs --version 3.7.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.7.0 --reuse-values \
--set mayastor.image.pullPolicy="IfNotPresent" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.io_engine.logLevel="info"
```
- For Upgrade from Local PV ZFS, use

Sample helm upgrade command from 3.4.1 to 3.7.0 or above versions:
```
helm upgrade openebs-mayastor openebs/openebs -n openebs --version 3.7.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.7.0 --reuse-values \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.io_engine.logLevel="info"
--set zfs-localpv.crds.zfsLocalPv.enabled=false
```
:::

3. Verify that the CRDs, Volumes, Snapshots and StoragePools are unaffected by the upgrade process.

## Replicated Storage

This section describes the Replicated Storage upgrade from OpenEBS Umbrella chart 3.x to OpenEBS 4.0.0.

1. Start the helm upgrade process with the new chart, i.e. 4.0.0 by using the below command:

Sample helm upgrade command from 3.5.0 to 3.7.0 or above versions:
```
helm upgrade openebs-mayastor openebs/openebs -n openebs --version 3.7.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.7.0 --reuse-values \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane="" \
--set mayastor.image.repoTags.controlPlane=""
```
The release versions 3.6.0 and above do not require special options to upgrade to 3.7.0 or above versions. The default upgrade command, should work here:
```
helm upgrade openebs-mayastor openebs/openebs -n openebs --version 3.8.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.8.0 --reuse-values
helm upgrade openebs openebs/openebs -n openebs --reuse-values \
--set localpv-provisioner.release.version=4.0.0 \
--set localpv-provisioner.localpv.image.tag=4.0.0 \
--set localpv-provisioner.helperPod.image.tag=4.0.0 \
--set mayastor.loki-stack.loki.image.tag=2.6.1 \
--set mayastor.loki-stack.filebeat.imageTag=7.17.3 \
--set mayastor.loki-stack.logstash.imageTag=1.0.1 \
--set mayastor.loki-stack.grafana.downloadDashboardsImage.tag=7.85.0 \
--set mayastor.loki-stack.grafana.image.tag=8.3.5 \
--set mayastor.loki-stack.grafana.sidecar.image.tag=1.19.2 \
--set mayastor.loki-stack.prometheus.alertmanager.image.tag=v0.23.0 \
--set mayastor.loki-stack.prometheus.nodeExporter.image.tag=v1.3.0 \
--set mayastor.loki-stack.prometheus.pushgateway.image.tag=v1.4.2 \
--set mayastor.loki-stack.prometheus.server.image.tag=v2.34.0 \
--set-json 'mayastor.loki-stack.promtail.config.clients[0]={"url": "http://{{ .Release.Name }}-loki:3100/loki/api/v1/push"}' \
--set mayastor.image.tag="v2.6.0" \
--set mayastor.csi.image.provisionerTag=v3.5.0 \
--set mayastor.csi.image.attacherTag=v4.3.0 \
--set mayastor.csi.image.snapshotterTag=v6.3.3 \
--set mayastor.csi.image.snapshotControllerTag=v6.3.3 \
--set mayastor.csi.image.registrarTag=v2.10.0 \
--set mayastor.crds.enabled=false \
--set-json 'mayastor.loki-stack.promtail.initContainer=[]'
```

Visit the mayastor chart configuration for more configuration options.
Add the options for the other storage engines (if any) to this command.
2. Verify that the CRDs, Volumes, Snapshots and StoragePools are unaffected by the upgrade process.

3. Start the Replicated Storage upgrade process by using the kubectl mayastor plugin v2.6.0.

:::note
It is a good idea to throw in the `--atomic` helm upgrade flag, so that the cluster may be reinstated to its source version, in case the upgrade to the target version fails. E.g.:
```
helm upgrade openebs-mayastor openebs/openebs -n openebs --version 3.8.0 --set mayastor.image.tag=$IMG_TAG --set release.version=3.8.0 --reuse-values --atomic
kubectl mayastor upgrade -n openebs
```
:::

#### 3. Execute the helm upgrade command
- This deploys an upgrade process of K8s resource type Job.

Execute the helm upgrade command. Proceed if and only if the command succeeds and all of the Mayastor Pods are ready. Check for Pod status using `kubectl get pods -n openebs`
```
kubectl get jobs -n openebs

#### 4. Follow the upgrade instructions for the mayastor helm chart
NAME COMPLETIONS DURATION AGE
openebs-upgrade-v2-6-0 1/1 4m49s 6m11s
```

Follow the upgrade instruction for the mayastor helm chart -> [https://mayastor.gitbook.io/introduction/additional-information/upgrade](https://mayastor.gitbook.io/introduction/additional-information/upgrade).
- Wait for the upgrade job to complete.

## Supported upgrade paths
```
kubectl get pods -n openebs

To upgrade to the latest version from your current version, you have to follow the below upgrade path.
- From 1.12.0 and higher to 3.6.0 - Get the steps from [here](https://github.com/openebs/upgrade/blob/develop/docs/upgrade.md).
openebs-upgrade-v2-6-0-s58xl 0/1 Completed 0 7m4s
```

4. Once the upgrade process completes, all the volumes and pools should be online.

## See Also:
## See Also

[See Release Notes](/introduction/releases) [Join our Community](/introduction/community) [Checkout Troubleshooting guides](/troubleshooting)
- [Release Notes](../releases.md)
- [Troubleshooting](../troubleshooting/troubleshooting-local-engine.md)
- [Join our Community](../community.md)
Loading