Skip to content

Commit

Permalink
[flunetd-elasticsearch] updated chart requirements - helm to 3.0.0 an…
Browse files Browse the repository at this point in the history
…d k8s to 1.16.0 (#22)

* updated chart requirements - helm to 3.0.0 and k8s to 1.16.0

Signed-off-by: André Bauer <[email protected]>

* use chart testing action v1.1.0 for lint

Signed-off-by: André Bauer <[email protected]>

* fixed readme

Signed-off-by: André Bauer <[email protected]>

* fix maintainer

Signed-off-by: André Bauer <[email protected]>

* added some links to the readme

Signed-off-by: André Bauer <[email protected]>

* fix markdown lint

Signed-off-by: André Bauer <[email protected]>

* sort plugins

Signed-off-by: André Bauer <[email protected]>
  • Loading branch information
monotek authored Nov 4, 2020
1 parent e9070d6 commit 1b48118
Show file tree
Hide file tree
Showing 16 changed files with 79 additions and 143 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
run: git fetch --prune --unshallow

- name: Run chart-testing (lint)
uses: helm/chart-testing-action@master
uses: helm/chart-testing-action@v1.1.0
with:
command: lint
config: .github/ct.yaml
Expand Down
8 changes: 5 additions & 3 deletions charts/fluentd-elasticsearch/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
apiVersion: v1
apiVersion: v2
name: fluentd-elasticsearch
version: 10.2.1
version: 11.0.0
appVersion: 3.1.0
type: application
home: https://www.fluentd.org/
description: A Fluentd Helm chart for Kubernetes with Elasticsearch output
icon: https://raw.githubusercontent.com/fluent/fluentd-docs/master/public/logo/Fluentd_square.png
Expand All @@ -18,5 +19,6 @@ sources:
- https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image
maintainers:
- name: monotek
email: andre.bauer@kiwigrid.com
email: monotek23@gmail.com
engine: gotpl
kubeVersion: ">=1.16.0-0"
19 changes: 15 additions & 4 deletions charts/fluentd-elasticsearch/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,19 @@ helm install kokuwa/fluentd-elasticsearch
## Introduction

This chart bootstraps a [Fluentd](https://www.fluentd.org/) daemonset on a [Kubernetes](http://kubernetes.io) cluster using the [Helm](https://helm.sh) package manager.

It's meant to be a drop in replacement for fluentd-gcp on GKE which sends logs to Google's Stackdriver service, but can also be used in other places where logging to ElasticSearch is required.
The used Docker image also contains Google's detect exceptions (for Java multiline stacktraces), Prometheus exporter, Kubernetes metadata filter & Systemd plugins.

The used [Docker](https://docker.com) image ([Dockerfile](https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image)) also contains the following plugins:

- [Detect exceptions](https://github.com/GoogleCloudPlatform/fluent-plugin-detect-exceptions) (for Java multiline stacktraces)
- [Kubernetes metadata filter](https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter)
- [Prometheus exporter](https://github.com/fluent/fluent-plugin-prometheus)
- [Systemd](https://github.com/fluent-plugin-systemd/fluent-plugin-systemd)

## Prerequisites

- Kubernetes 1.8+ with Beta APIs enabled
- Kubernetes 1.16+ with Beta APIs enabled

## Installing the Chart

Expand All @@ -31,10 +38,10 @@ The command deploys fluentd-elasticsearch on the Kubernetes cluster in the defau

## Uninstalling the Chart

To uninstall/delete the `my-release` deployment:
To uninstall the `my-release` deployment:

```console
helm delete my-release
helm uninstall my-release
```

The command removes all the Kubernetes components associated with the chart and deletes the release.
Expand Down Expand Up @@ -328,3 +335,7 @@ In this version elasticsearch template in `output.conf` configmap was expanded t
- `buffer.chunkLimitSize`
- `buffer.queueLimitLength`
- `buffer.overflowAction`

### From a version < 10.0.0 to version => 11.0.0

The chart requires now Helm >= 3.0.0 and Kubernetes >= 1.16.0
67 changes: 37 additions & 30 deletions charts/fluentd-elasticsearch/templates/_helpers.tpl
Original file line number Diff line number Diff line change
@@ -1,55 +1,62 @@
{{/* vim: set filetype=mustache: */}}
{{/*
Expand the name of the chart.
*/}}
{{- define "fluentd-elasticsearch.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "fluentd-elasticsearch.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- $name := default .Chart.Name .Values.nameOverride -}}
{{- if contains $name .Release.Name -}}
{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}
{{- end -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "fluentd-elasticsearch.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create the name of the service account to use
*/}}
{{- define "fluentd-elasticsearch.serviceAccountName" -}}
{{- if .Values.serviceAccount.create -}}
{{ default (include "fluentd-elasticsearch.fullname" .) .Values.serviceAccount.name }}
{{- else -}}
{{ default "default" .Values.serviceAccount.name }}
{{- end -}}
{{- end -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
Common labels
*/}}
{{- define "fluentd-elasticsearch.labels" -}}
app.kubernetes.io/name: {{ include "fluentd-elasticsearch.name" . }}
helm.sh/chart: {{ include "fluentd-elasticsearch.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{ include "fluentd-elasticsearch.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}
{{- end }}

{{/*
Selector labels
*/}}
{{- define "fluentd-elasticsearch.selectorLabels" -}}
app.kubernetes.io/name: {{ include "fluentd-elasticsearch.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Create the name of the service account to use
*/}}
{{- define "fluentd-elasticsearch.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "fluentd-elasticsearch.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}
6 changes: 1 addition & 5 deletions charts/fluentd-elasticsearch/templates/clusterrole.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,10 @@
{{- if .Values.rbac.create -}}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ include "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
rules:
- apiGroups:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,10 @@
{{- if .Values.rbac.create -}}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: {{ include "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
subjects:
- kind: ServiceAccount
Expand Down
7 changes: 1 addition & 6 deletions charts/fluentd-elasticsearch/templates/configmaps.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,9 @@

{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
data:
{{- if .Values.configMaps.useDefaults.systemConf }}
Expand Down
29 changes: 3 additions & 26 deletions charts/fluentd-elasticsearch/templates/daemonset.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,9 @@

{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
{{- if semverCompare ">= 1.9" $striped_version -}}
apiVersion: apps/v1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: DaemonSet
metadata:
name: {{ include "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
{{- if .Values.annotations }}
annotations:
Expand All @@ -23,24 +14,12 @@ spec:
{{ toYaml .Values.updateStrategy | indent 4 }}
selector:
matchLabels:
app.kubernetes.io/name: {{ include "fluentd-elasticsearch.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- include "fluentd-elasticsearch.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 8 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.selectorLabels" . | nindent 8 }}
annotations:
{{- if semverCompare "< 1.13" $striped_version }}
# This annotation ensures that fluentd does not get evicted if the node
# supports critical pod annotation based priority scheme.
# Note that this does not guarantee admission on the nodes (#40573).
# NB! this annotation is deprecated as of version 1.13 and will be removed in 1.14.
# ref: https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/
scheduler.alpha.kubernetes.io/critical-pod: ''
{{- end }}
checksum/config: {{ include (print $.Template.BasePath "/configmaps.yaml") . | sha256sum }}
{{- if .Values.podAnnotations }}
{{ toYaml .Values.podAnnotations | indent 8 }}
Expand Down Expand Up @@ -175,9 +154,7 @@ spec:
- name: K8S_NODE_NAME
valueFrom:
fieldRef:
{{- if semverCompare ">= 1.9" $striped_version }}
apiVersion: v1
{{- end }}
fieldPath: spec.nodeName
resources:
{{ toYaml .Values.resources | indent 10 }}
Expand Down
10 changes: 2 additions & 8 deletions charts/fluentd-elasticsearch/templates/metrics-service.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,10 @@
{{- if or (.Values.serviceMonitor.enabled) (.Values.serviceMetric.enabled) }}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "fluentd-elasticsearch.fullname" $ }}-metrics
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
spec:
type: {{ .Values.serviceMonitor.type }}
Expand All @@ -18,6 +13,5 @@ spec:
port: {{ .Values.serviceMonitor.port }}
targetPort: {{ .Values.serviceMonitor.port }}
selector:
app.kubernetes.io/name: {{ include "fluentd-elasticsearch.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- include "fluentd-elasticsearch.selectorLabels" . | nindent 4 }}
{{- end }}
10 changes: 1 addition & 9 deletions charts/fluentd-elasticsearch/templates/pod-security-policy.yaml
Original file line number Diff line number Diff line change
@@ -1,18 +1,10 @@
{{- if .Values.podSecurityPolicy.enabled }}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
{{- if semverCompare "> 1.15" $striped_version -}}
apiVersion: policy/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: PodSecurityPolicy
metadata:
name: {{ template "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
annotations:
{{- if .Values.podSecurityPolicy.annotations }}
Expand Down
6 changes: 1 addition & 5 deletions charts/fluentd-elasticsearch/templates/prometheusrule.yaml
Original file line number Diff line number Diff line change
@@ -1,14 +1,10 @@
{{- if .Values.prometheusRule.enabled }}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: {{ template "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
{{- if .Values.prometheusRule.labels }}
{{- toYaml .Values.prometheusRule.labels | nindent 4 }}
Expand Down
10 changes: 1 addition & 9 deletions charts/fluentd-elasticsearch/templates/role.yaml
Original file line number Diff line number Diff line change
@@ -1,21 +1,13 @@
{{- if .Values.podSecurityPolicy.enabled }}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ template "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
rules:
{{- if semverCompare "> 1.15" $striped_version }}
- apiGroups: ['policy']
{{- else }}
- apiGroups: ['extensions']
{{- end }}
resources: ['podsecuritypolicies']
verbs: ['use']
resourceNames:
Expand Down
6 changes: 1 addition & 5 deletions charts/fluentd-elasticsearch/templates/rolebinding.yaml
Original file line number Diff line number Diff line change
@@ -1,15 +1,11 @@

{{- if .Values.podSecurityPolicy.enabled }}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ template "fluentd-elasticsearch.fullname" . }}
labels:
{{ include "fluentd-elasticsearch.labels" . | indent 4 }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
roleRef:
kind: Role
Expand Down
17 changes: 5 additions & 12 deletions charts/fluentd-elasticsearch/templates/service-account.yaml
Original file line number Diff line number Diff line change
@@ -1,20 +1,13 @@
{{- if .Values.serviceAccount.create -}}
{{- $striped_version := (split "-" .Capabilities.KubeVersion.GitVersion)._0 -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ if .Values.serviceAccount.name }}{{ .Values.serviceAccount.name }}{{ else }}{{ include "fluentd-elasticsearch.fullname" . }}{{ end }}
name: {{ include "fluentd-elasticsearch.serviceAccountName" . }}
labels:
app.kubernetes.io/name: {{ include "fluentd-elasticsearch.name" . }}
helm.sh/chart: {{ include "fluentd-elasticsearch.chart" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- if semverCompare "< 1.6" $striped_version }}
kubernetes.io/cluster-service: "true"
{{- end }}
{{- include "fluentd-elasticsearch.labels" . | nindent 4 }}
addonmanager.kubernetes.io/mode: Reconcile
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- if .Values.serviceAccount.annotations }}
{{ toYaml .Values.serviceAccount.annotations | indent 4 }}
{{- end }}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end -}}
Loading

2 comments on commit 1b48118

@doutside
Copy link

@doutside doutside commented on 1b48118 Nov 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi,
I'm having troubles running chart since day before yesterday.

Install chart params
helm install -g kokuwa/fluentd-elasticsearch -f chart-fluentd-values.yaml -n logs --wait

The error
Error: template: fluentd-elasticsearch/templates/service.yaml:24:8: executing "fluentd-elasticsearch/templates/service.yaml" at <include "fluentd-elasticsearch.selectorLabels" .>: error calling include: template: fluentd-elasticsearch/templates/_helpers.tpl:49:27: executing "fluentd-elasticsearch.selectorLabels" at <include "fluentd-elasticsearch.name" .>: error calling include: template: fluentd-elasticsearch/templates/_helpers.tpl:5:18: executing "fluentd-elasticsearch.name" at <.Chart.Name>: nil pointer evaluating interface {}.Name

@monotek
Copy link
Member Author

@monotek monotek commented on 1b48118 Nov 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be fixed in: #24

Please sign in to comment.