kopia + rclone Docker image

Docker image with kopia and rclone

---

---

Table of Contents

1. About
2. Versions
   1. Supported architectures
3. Usage
   1. Examples
4. Build locally
5. Third party notices
6. License

---

About

This docker image contains kopia and rclone build from source which can be used out of the box. kopia is an open source backup tool with other useful features such as encryption, deduplication and compression. It has a rclone integration for storing the backups in one of the cloud storage systems supported by rclone.

Versions

You can find detailed information about the supported versions of kopia and rclone in the versions' wiki.

Usage

You can pull the latest version of the image with this command:

docker pull ghcr.io/infinityofspace/kopia_rclone_docker:latest

To use rclone, you must first create a configuration of the storage system to be used. The rclone configuration is in the /rclone path. If you have not already created an rclone configuration for the desired storage system you can use the following command and follow the rclone instructions to create such configuration:

docker run \
    -v $(pwd)/rclone:/rclone \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        rclone config

The usage of rclone can be found in the official documentation.

Now we use the rclone storage system configuration for kopia. You can use the kopia commands as normal:

docker run \
    -v $(pwd)/rclone:/rclone \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        kopia --version

You can also put all the kopia commands into a shell script and then run them in the docker container at startup (here the kopia_cmds.sh file contains all the commands, make sure the file is executable). Moreover, you can specify the kopia password with the KOPIA_PASSWORD environment variable:

docker run \
    -v $(pwd)/rclone:/rclone \
    -v $(pwd)/kopia_cmds.sh:/kopia_cmds.sh \
    -e KOPIA_PASSWORD=mysecretpassword \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        /kopia_cmds.sh

To store the kopia configuration independently of a docker container you can use docker volumes. The kopia configuration is in the /kopia/config path and the kopia cache is in /kopia/cache. Here we store the kopia configuration and data cache in docker volumes:

docker run \
    -v $(pwd)/rclone:/rclone \
    -v kopia_config:/kopia/config \
    -v kopia_cache:/kopia/cache \
    -e KOPIA_PASSWORD=mysecretpassword \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        kopia repository create rclone \
        --remote-path testcloud:/path/to/the/backup/dir

The following environment variables are defined for the docker image:

• KOPIA_CONFIG_PATH=/kopia/config/repository.config
• KOPIA_LOG_DIR=/kopia/logs
• KOPIA_CACHE_DIRECTORY=/kopia/cache
• RCLONE_CONFIG=/rclone/rclone.conf
• KOPIA_CHECK_FOR_UPDATES=false

You can override them if you want to adjust the paths or values.

The official usage documentation of kopia can be found here.

Examples

rclone or kopia can be used as usual. In the examples, we assume that we already have a valid rclone configuration in the current working directory. Some examples are listed below:

rclone

Get quota information of the remote mycloud:

docker run \
    -v $(pwd)/rclone:/rclone \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        rclone about mycloud:

List all content in the path /backups on the remote mycloud:

docker run \
    -v $(pwd)/rclone:/rclone \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        rclone ls mycloud:/backups

Show the version number of rclone:

docker run \
    -v $(pwd)/rclone:/rclone \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        rclone version

kopia

Initialise a new kopia repository in the remote path /backup/data on the remote mycloud. Additionally, the password used to encrypt the repo data is mysecretpassword and the kopia configuration respectively cache is written to the Docker volumes kopia_config and kopia_cache:

docker run \
    -v $(pwd)/rclone:/rclone \
    -v kopia_config:/kopia/config \
    -v kopia_cache:/kopia/cache \
    -e KOPIA_PASSWORD=mysecretpassword \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        kopia repository create rclone \
        --remote-path mycloud:/backup/data

Connect to an existing kopia repository in the remote path /backup/data on the remote mycloud. Additionally, the password used to encrypt the repo data is mysecretpassword and the kopia configuration respectively cache is written to the Docker volumes kopia_config and kopia_cache:

docker run \
    -v $(pwd)/rclone:/rclone \
    -v kopia_config:/kopia/config \
    -v kopia_cache:/kopia/cache \
    -e KOPIA_PASSWORD=mysecretpassword \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
        kopia repository connect rclone \
        --remote-path mycloud:/backup/data

Set the global policy for the number of kept annual snapshot of the currently connected repo to 12, where the currently connected repo is stored in the docker volume kopia_config:

docker run \
    -v $(pwd)/rclone:/rclone \
    -v kopia_config:/kopia/config \
    -v kopia_cache:/kopia/cache \
    -e KOPIA_PASSWORD=mysecretpassword \
    ghcr.io/infinityofspace/kopia_rclone_docker:latest \
      kopia \
      policy \
      set \
      --global \
      --keep-annual 12

Backup Docker volumes

The following example describes the backup of any Docker volume. Advantages of the backup with kopia and rclone:

• end to end encrypted backup
• duplicate the backup with kopia repository synchronization
• incremental backups
• send backup to remote storage (this means that if local storage space is limited, complete backups that require more storage than is available locally are still possible)

Docker standalone

The following example describes the backup of a Docker volume with kopia and rclone in a standalone Docker installation.

Before we get started, we need to customise the image. We need to add docker-cli to the base image, which we need to interact with the Docker socket. For this purpose, create the file Dockerfile with the following content:

FROM ghcr.io/infinityofspace/kopia_rclone_docker:latest

RUN apk add --no-cache docker-cli

In the next step, create the docker-compose.yml file with the following content:

services:
    kopia:
      build: .
      restart: unless-stopped
      ports:
          - 127.0.0.1:8080:8080
      environment:
          - KOPIA_PERSIST_CREDENTIALS_ON_CONNECT=true
      volumes:
          - kopia_config:/kopia/config
          - kopia_logs:/kopia/logs

          # rclone config folder
          - ./rclone_config:/rclone
          # custom docker container shutdown and startup scripts folder
          - ./scripts:/scripts

          # rootless docker socket location
          - "$XDG_RUNTIME_DIR/docker.sock:/var/run/docker.sock"
          # uncomment the next line when you use a rootfull docker socket location
          # - /var/run/docker.sock:/var/run/docker.sock

          # all volumes which should be backed up
          - volume1:/volumes/volume1:ro
        command:
          - kopia
          - server
          - start
          - --insecure
          - --address=0.0.0.0:8080
          - --server-username=${USER}
          - --server-password=${KOPIA_PASSWORD}
volumes:
    kopia_config:
    kopia_logs:
    volume1:
        external: true

The service kopia is defined in the docker compose file, which offers a web UI for creating and managing backups. This Web UI can be accessed at the address http://localhost:8080.

However, before you start the docker compose service, you must first create scripts for the shutdown and start docker container that use the desired volume. Create the container shutdown and restart scripts in a folder called scripts next to the docker compose file (the name of the container is in this example container_service1):

Shutdown script:

#!/bin/sh

docker stop $1

Start script:

#!/bin/sh

docker start $1

Savethe shutdown and start scripts in the scripts folder. After you have created the scripts, you still have to make them executable. This is possible under common Unix systems with this command:

chmod -x scripts/*

The last preparation step is to define the kopia user and password. This is done using the following .env file:

USER=
KOPIA_PASSWORD=

Finally, start the kopia docker service with the command:

docker compose up -d

---

Now open the address http://localhost:8080 of the Web UI in a web browser. At the first start, set up the kopia repository and select rclone as the provider. Also make sure the config file is set to /kopia/config/repository.config. In the next step, we set up the backup of volume volume1 by entering the path /volumes/volume1 of volume volume1 of the container under the menu item Policies in the input field enter directory to find or set policy and creating the policy with the button Set Policy. To stop the container in the backup process so that no unstable data state of the backup occurs, we must now set up the shutdown and start scripts in the previously created policy. To do this, use the Edit button of the corresponding policy in the Policies menu item. Now enter the script path shutdown.sh with the contains names as arguments in the Before Snapshot entry of the Snapshot Action menu. For the entry After Snapshot enter start.sh with the contains names as arguments and save the policy with the Save Policy button.

At this point, the Docker volume volume1 can be backed up. Ideally, set up a scheduled backup in the policy for the path so that a backup is performed automatically.

Build locally

Before you create the image you must first clone the repo:

git clone https://github.com/infinityofspace/kopia_rclone_docker.git

Change the current working directory:

cd kopia_rclone_docker

You can build the image yourself locally instead of using the prebuild image. Use the following command to build the docker image with a full kopia server version. This version includes the web gui and is built directly from the master branches of kopia and rclone:

docker build -t kopia_rclone_docker .

If you don't need the web gui of kopia server, then you can specify this with the build argument KOPIA_BUILD_TYPE=noui and kopia will be built without the web gui:

docker build \
    --build-arg KOPIA_BUILD_TYPE=noui \
    -t kopia_rclone_docker \
    .

If you want to use a specific version of kopia or rlcone for the image build, you can do this with the environment variables RCLONE_BRANCH and KOPIA_BRANCH respectively:

docker build \
    --build-arg KOPIA_BUILD_TYPE=noui \
    --build-arg RCLONE_BRANCH=v1.56.0 \
    --build-arg KOPIA_BRANCH=v0.8.4 \
    -t kopia_rclone_docker \
    .

Third party notices

This project uses source code with different licenses, these are listed in the following:

• rclone:
  • link to project
  • link to license
• kopia:
  • link to project
  • link to license

By using this docker image, you agree to the listed licenses.

License

MIT - Copyright (c) 2021 Marvin Heptner