Skip to content
This repository was archived by the owner on Nov 3, 2022. It is now read-only.

Commit

Permalink
feat: updated dockerfile for using base os as ubuntu & recent version…
Browse files Browse the repository at this point in the history
…s of different components
  • Loading branch information
priyesh2609 committed Nov 23, 2021
1 parent 10f88c4 commit 4a6e90c
Show file tree
Hide file tree
Showing 10 changed files with 194 additions and 201 deletions.
58 changes: 25 additions & 33 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.15
FROM golang:1.16-buster

ARG VERSION

Expand All @@ -12,17 +12,18 @@ RUN CGO_ENABLED=0 GOOS=linux \
-a -installsuffix cgo \
-o mgob github.com/stefanprodan/mgob/cmd/mgob

FROM alpine:3.12
FROM ubuntu:focal

ARG BUILD_DATE
ARG VCS_REF
ARG VERSION

ENV MONGODB_TOOLS_VERSION 4.2.3-r1
ENV GNUPG_VERSION 2.2.23-r0
ENV MONGODB_TOOLS_VERSION 100.5.1
ENV MONGODB_VERSION 4.4
ENV GNUPG_VERSION 2.2.27
ENV GOOGLE_CLOUD_SDK_VERSION 316.0.0
ENV AZURE_CLI_VERSION 2.13.0
ENV AWS_CLI_VERSION 1.18.159
ENV AWS_CLI_VERSION 2.0.30
ENV PATH /root/google-cloud-sdk/bin:$PATH

LABEL org.label-schema.build-date=$BUILD_DATE \
Expand All @@ -35,7 +36,22 @@ LABEL org.label-schema.build-date=$BUILD_DATE \
org.label-schema.version=$VERSION \
org.label-schema.schema-version="1.0"

RUN apk add --no-cache ca-certificates tzdata mongodb-tools=${MONGODB_TOOLS_VERSION} gnupg=${GNUPG_VERSION}
RUN apt-get update && apt install apt-transport-https ca-certificates gnupg tzdata wget unzip curl -y
RUN wget -qO - https://www.mongodb.org/static/pgp/server-${MONGODB_VERSION}.asc | apt-key add -
RUN echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/${MONGODB_VERSION} multiverse" | tee /etc/apt/sources.list.d/mongodb-org-${MONGODB_VERSION}.list
RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -

RUN apt update && apt install -y google-cloud-sdk
RUN gcloud config set core/disable_usage_reporting true && \
gcloud config set component_manager/disable_update_check true && \
gcloud config set metrics/environment github_docker_image && \
gcloud --version

RUN wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.5.1.tgz
RUN tar -zxvf mongodb-database-tools-*-100.5.1.tgz
RUN cp /mongodb-database-tools-*-100.5.1/bin/* /usr/bin/

ADD https://dl.minio.io/client/mc/release/linux-amd64/mc /usr/bin
RUN chmod u+x /usr/bin/mc

Expand All @@ -47,35 +63,11 @@ RUN cd /tmp \

WORKDIR /root/

#install gcloud
# https://github.com/GoogleCloudPlatform/cloud-sdk-docker/blob/69b7b0031d877600a9146c1111e43bc66b536de7/alpine/Dockerfile
RUN apk --no-cache add \
curl \
python3 \
py3-pip \
bash \
libc6-compat \
openssh-client \
git \
&& pip3 --no-cache-dir install --upgrade pip && \
pip --no-cache-dir install wheel && \
pip --no-cache-dir install crcmod && \
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${GOOGLE_CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
tar xzf google-cloud-sdk-${GOOGLE_CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
rm google-cloud-sdk-${GOOGLE_CLOUD_SDK_VERSION}-linux-x86_64.tar.gz && \
ln -s /lib /lib64 && \
gcloud config set core/disable_usage_reporting true && \
gcloud config set component_manager/disable_update_check true && \
gcloud config set metrics/environment github_docker_image && \
gcloud --version

# install azure-cli and aws-cli
RUN apk --no-cache add --virtual=build gcc libffi-dev musl-dev openssl-dev python3-dev make && \
pip --no-cache-dir install cffi && \
pip --no-cache-dir --use-feature=2020-resolver install azure-cli==${AZURE_CLI_VERSION} && \
pip --no-cache-dir install awscli==${AWS_CLI_VERSION} && \
apk del --purge build
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64-${AWS_CLI_VERSION}.zip" -o "awscliv2.zip" && unzip awscliv2.zip && ./aws/install

RUN apt autoremove -y
COPY --from=0 /go/src/github.com/stefanprodan/mgob/mgob .

VOLUME ["/config", "/storage", "/tmp", "/data"]
Expand Down
58 changes: 29 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,34 +7,34 @@ MGOB is a MongoDB backup automation tool built with Go.

#### Features

* schedule backups
* local backups retention
* upload to S3 Object Storage (Minio, AWS, Google Cloud, Azure)
* upload to gcloud storage
* upload to SFTP
* upload to any [Rclone](https://rclone.org/) supported storage
* notifications (Email, Slack)
* instrumentation with Prometheus
* http file server for local backups and logs
* distributed as an Alpine Docker image
- schedule backups
- local backups retention
- upload to S3 Object Storage (Minio, AWS, Google Cloud, Azure)
- upload to gcloud storage
- upload to SFTP
- upload to any [Rclone](https://rclone.org/) supported storage
- notifications (Email, Slack)
- instrumentation with Prometheus
- http file server for local backups and logs
- distributed as an Alpine Docker image

#### Install

MGOB is available on Docker Hub at [stefanprodan/mgob](https://hub.docker.com/r/stefanprodan/mgob/).

Supported tags:

* `stefanprodan/mgob:latest` latest stable [release](https://github.com/stefanprodan/mgob/releases)
* `stefanprodan/mgob:edge` master branch latest successful [build](https://travis-ci.org/stefanprodan/mgob)
- `stefanprodan/mgob:latest` latest stable [release](https://github.com/stefanprodan/mgob/releases)
- `stefanprodan/mgob:edge` master branch latest successful [build](https://travis-ci.org/stefanprodan/mgob)

Compatibility matrix:

MGOB| MongoDB
-----|--------
`stefanprodan/mgob:0.9` | 3.4
`stefanprodan/mgob:0.10` | 3.6
`stefanprodan/mgob:1.0` | 4.0
`stefanprodan/mgob:1.1` | 4.2
| MGOB | MongoDB |
| ------------------------ | ------- |
| `stefanprodan/mgob:0.9` | 3.4 |
| `stefanprodan/mgob:0.10` | 3.6 |
| `stefanprodan/mgob:1.0` | 4.0 |
| `stefanprodan/mgob:1.1` | 4.2 |

Docker:

Expand Down Expand Up @@ -90,7 +90,7 @@ encryption:
keyServer: hkps://keys.openpgp.org
# optional list of recipients, they will be looked up on key server
recipients:
- [email protected]
- [email protected]
# S3 upload (optional)
s3:
url: "https://play.minio.io:9000"
Expand All @@ -99,12 +99,12 @@ s3:
accessKey: "Q3AM3UQ867SPQQA43P2F"
secretKey: "zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"
# Optional, only used for AWS (when awscli is present)
# The customer-managed AWS Key Management Service (KMS) key ID that should be used to
# The customer-managed AWS Key Management Service (KMS) key ID that should be used to
# server-side encrypt the backup in S3
#kmsKeyId:
# Optional, only used for AWS (when awscli is present)
# Valid choices are: STANDARD | REDUCED_REDUNDANCY | STANDARD_IA | ONE-
# ZONE_IA | INTELLIGENT_TIERING | GLACIER | DEEP_ARCHIVE.
# ZONE_IA | INTELLIGENT_TIERING | GLACIER | DEEP_ARCHIVE.
# Defaults to 'STANDARD'
#storageClass: STANDARD
# For Minio and AWS use S3v4 for GCP use S3v2
Expand Down Expand Up @@ -176,15 +176,15 @@ target:
#### Web API
* `mgob-host:8090/storage` file server
* `mgob-host:8090/status` backup jobs status
* `mgob-host:8090/metrics` Prometheus endpoint
* `mgob-host:8090/version` mgob version and runtime info
* `mgob-host:8090/debug` pprof endpoint
- `mgob-host:8090/storage` file server
- `mgob-host:8090/status` backup jobs status
- `mgob-host:8090/metrics` Prometheus endpoint
- `mgob-host:8090/version` mgob version and runtime info
- `mgob-host:8090/debug` pprof endpoint

On demand backup:

* HTTP POST `mgob-host:8090/backup/:planID`
- HTTP POST `mgob-host:8090/backup/:planID`

```bash
curl -X POST http://mgob-host:8090/backup/mongo-debug
Expand All @@ -202,8 +202,8 @@ curl -X POST http://mgob-host:8090/backup/mongo-debug

Scheduler status:

* HTTP GET `mgob-host:8090/status`
* HTTP GET `mgob-host:8090/status/:planID`
- HTTP GET `mgob-host:8090/status`
- HTTP GET `mgob-host:8090/status/:planID`

```bash
curl -X GET http://mgob-host:8090/status/mongo-debug
Expand Down
6 changes: 3 additions & 3 deletions chart/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: |
name: mgob
version: 1.0.0
sources:
- https://github.com/stefanprodan/mgob
- https://github.com/stefanprodan/mgob
maintainers:
- name: endrec
email: [email protected]
- name: endrec
email: [email protected]
47 changes: 24 additions & 23 deletions chart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,33 +7,34 @@ Runs scheduled backups with retention, S3 & SFTP upload, notifications, instrume

## Maintainers

| Name | Email | Url |
| ---- | ------ | --- |
| endrec | [email protected] | |
| Name | Email | Url |
| ------ | -------------------------- | --- |
| endrec | [email protected] | |

## Source Code

* <https://github.com/stefanprodan/mgob>
- <https://github.com/stefanprodan/mgob>

## Values

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| config | object | `{}` | Backup plans. For details, see [values.yaml](values.yaml) |
| env | object | `{}` | |
| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
| image.repository | string | `"stefanprodan/mgob"` | Image repo |
| image.tag | float | `1.1` | Image tag |
| replicaCount | int | `1` | Number of replicas |
| resources | object | `{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}` | Resource requests and limits ref: http://kubernetes.io/docs/user-guide/compute-resources/ |
| secret | object | `{}` | Secret(s) to mount. For details, see [values.yaml](values.yaml) |
| service.externalPort | int | `8090` | Port to access the service |
| service.internalPort | int | `8090` | Port to connect to in pod |
| service.name | string | `"mgob"` | Service name |
| serviceAccount.annotations | object | `{}` | Annotations to add on service account |
| serviceAccount.create | bool | `true` | If false, default service account will be used |
| storage.longTerm | object | `{"accessMode":"ReadWriteOnce","name":"mgob-storage","size":"10Gi","storageClass":"gp2"}` | Persistent volume for backups, see `config.retention` |
| storage.tmp | object | `{"accessMode":"ReadWriteOnce","name":"mgob-tmp","size":"3Gi","storageClass":"gp2"}` | Persistent volume for temporary files |

----------------------------------------------
| Key | Type | Default | Description |
| -------------------------- | ------ | ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| config | object | `{}` | Backup plans. For details, see [values.yaml](values.yaml) |
| env | object | `{}` | |
| image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
| image.repository | string | `"stefanprodan/mgob"` | Image repo |
| image.tag | float | `1.1` | Image tag |
| replicaCount | int | `1` | Number of replicas |
| resources | object | `{"limits":{"cpu":"100m","memory":"128Mi"},"requests":{"cpu":"100m","memory":"128Mi"}}` | Resource requests and limits ref: http://kubernetes.io/docs/user-guide/compute-resources/ |
| secret | object | `{}` | Secret(s) to mount. For details, see [values.yaml](values.yaml) |
| service.externalPort | int | `8090` | Port to access the service |
| service.internalPort | int | `8090` | Port to connect to in pod |
| service.name | string | `"mgob"` | Service name |
| serviceAccount.annotations | object | `{}` | Annotations to add on service account |
| serviceAccount.create | bool | `true` | If false, default service account will be used |
| storage.longTerm | object | `{"accessMode":"ReadWriteOnce","name":"mgob-storage","size":"10Gi","storageClass":"gp2"}` | Persistent volume for backups, see `config.retention` |
| storage.tmp | object | `{"accessMode":"ReadWriteOnce","name":"mgob-tmp","size":"3Gi","storageClass":"gp2"}` | Persistent volume for temporary files |

---

Autogenerated from chart metadata using [helm-docs v1.3.0](https://github.com/norwoodj/helm-docs/releases/v1.3.0)
10 changes: 5 additions & 5 deletions chart/notes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Usage
=====
# Usage

This script assumes a user has been created in the mongoDB instance with sufficient read privileges to create the
backups.

Expand All @@ -13,15 +13,15 @@ those match, or your backups will fail.
{ role: "backup", db: "admin" }
]
});

Secondly, it assumes that [Helm](https://github.com/kubernetes/helm) is installed on your cluster and that you have
a properly configured [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/) installed.

To install the chart, first clone the Git repository. Secondly, edit the `values.yaml`-file to define your backup
plans. Documentation for those can be found in the default repository `readme.md`, easily accessible as the
[mgob repository start page](https://github.com/stefanprodan/mgob). When the `values.yaml`-file properly represents the
[mgob repository start page](https://github.com/stefanprodan/mgob). When the `values.yaml`-file properly represents the
plan(s) you want to create, simply run:

$ helm install --namespace my-kubernetes-ns ./chart

This will install the chart on your cluster.
This will install the chart on your cluster.
4 changes: 2 additions & 2 deletions chart/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,13 @@ storage:
# -- Persistent volume for backups, see `config.retention`
longTerm:
accessMode: "ReadWriteOnce"
storageClass: "gp2" # Note: "gp2" is for AWS. Use the storage class for your cloud provider.
storageClass: "gp2" # Note: "gp2" is for AWS. Use the storage class for your cloud provider.
name: "mgob-storage"
size: 10Gi
# -- Persistent volume for temporary files
tmp:
accessMode: "ReadWriteOnce"
storageClass: "gp2" # Note: "gp2" is for AWS. Use the storage class for your cloud provider.
storageClass: "gp2" # Note: "gp2" is for AWS. Use the storage class for your cloud provider.
name: "mgob-tmp"
size: 3Gi

Expand Down
7 changes: 4 additions & 3 deletions cmd/mgob/mgob.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,12 @@ func main() {
},
cli.IntFlag{
Name: "Port,p",
Usage: "HTTP port to listen on",
Usage: "Port to bind the HTTP server on",
Value: 8090,
},
cli.StringFlag{
Name: "Host,h",
Usage: "HTTP host to listen on",
Name: "Bind,b",
Usage: "Host to bind the HTTP server on",
Value: "",
},
cli.BoolFlag{
Expand All @@ -95,6 +95,7 @@ func start(c *cli.Context) error {
appConfig.LogLevel = c.String("LogLevel")
appConfig.JSONLog = c.Bool("JSONLog")
appConfig.Port = c.Int("Port")
appConfig.Host = c.String("Bind")
appConfig.ConfigPath = c.String("ConfigPath")
appConfig.StoragePath = c.String("StoragePath")
appConfig.TmpPath = c.String("TmpPath")
Expand Down
Loading

0 comments on commit 4a6e90c

Please sign in to comment.