Skip to content

Commit

Permalink
initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
mouchar committed Jan 7, 2025
0 parents commit ae54c08
Show file tree
Hide file tree
Showing 9 changed files with 526 additions and 0 deletions.
190 changes: 190 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
# Istio with GD.CN

GoodData CN is almost ready for running on clusters with Istio Gateway (in sidecar mode), so you don't need to
use Nginx Ingress controller.

Recent helm charts to not require any modifications, but a few things need to be taken into account when using Istio:

* port names: gooddata-cn services follow Istio recommendation on naming ports. However, etcd and pulsar charts do not.
Fortunately it is possible to override default port names. Refer to [gdcn-values.yaml](gdcn-values.yaml) and
[pulsar-values.yaml](pulsar-values.yaml) and see how to add `tcp-` prefix to ports and make Istio protocol discovery happy.
* gooddata-cn defines `nginx` as default `ingressClassName`. I assume this ingress class is NOT available on your istio-enabled
cluster. This is expected until we update our apps to support Istio's custom resources natively. So when you create a new Organization,
a new Ingress will be created but will not be used (because of unregistered ingress class name). You will need to create VirtualService
for your organization manually.


## Requirements for local test

* Running Docker daemon
* curl
* [KinD binary](https://kind.sigs.k8s.io/docs/user/quick-start/)
* [cloud-provider-kind](https://kind.sigs.k8s.io/docs/user/loadbalancer/)
* Valid GoodData CN License key, stored in `GDCN_LICENSE` env variable
* [istioctl binary](https://github.com/istio/istio/releases)
* [kubectl](https://kubernetes.io/docs/tasks/tools/)
* Optionally [helm binary](https://helm.sh), if you want Kiali UI for Istio

## Setup

1. Create KIND cluster

```bash
kind create cluster --name kind
# get it from https://github.com/kubernetes-sigs/cloud-provider-kind/releases
cloud-provider-kind -v 0 &
```

1. Install Istio and related stuff
I tested with "native sidecar" mode, so it requires k8s 1.29+. It resolves strange issues with jobs not being terminated.

```bash
istioctl install --set values.pilot.env.ENABLE_NATIVE_SIDECARS=true -y --set meshConfig.accessLogFile=/dev/stdout
# These are optional but recommended for better visiblity
kubectl apply -f https://raw.githubusercontent.com/istio/istio/1.24.2/samples/addons/prometheus.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/1.24.2/samples/addons/grafana.yaml
helm upgrade --install -n istio-system kiali-server --repo https://kiali.org/helm-charts kiali-server --set auth.strategy=anonymous
```

1. Install Apache Pulsar and GoodData CN

```bash
kubectl apply -f namespaces.yaml

kind load docker-image apachepulsar/pulsar:3.3.3
helm -n pulsar upgrade --install \
--repo https://pulsar.apache.org/charts pulsar pulsar --version 3.5.0 \
--values pulsar-values.yaml

# GD CN License key
kubectl -n gooddata create secret generic gdcn-license --from-literal=license=$GDCN_LICENSE

# provision certificate and key for *.example.com, keep in files _.example.com.crt and _.example.com.key
# I'm using my own local CA, so cacert should be passed to k8s secret as well.

# secrets must be stored in istio-system so SDS can find them (?)
kubectl -n istio-system create secret generic star.example.com \
--from-file=cert=_.example.com.crt \
--from-file=key=_.example.com.key \
--from-file=cacert=ca.crt


# Install official gooddata chart
helm -n gooddata upgrade --install \
--repo https://charts.gooddata.com/ \
gooddata-cn gooddata-cn --version 3.25.0 \
--values gdcn-values.yaml

# OR from local chart files
helm -n gooddata upgrade --install \
gooddata-cn ./folder-with-extracted-gooddata-cn-chart \
--values gdcn-values.yaml --set image.defaultTag=3.25.0

```

1. Create Istio Ingress GW in "gooddata" namespace.

```bash
kubectl apply -f gateway.yaml
```

1. Create VirtualService for Dex

```bash
kubectl apply -f istio-virtual-service-dex.yaml
```

1. Create delegate VS (shared VS without hosts or attached gateway)

```bash
kubectl apply -f istio-virtual-service.yaml
```

1. Update `/etc/hosts` with example hostnames.
This guide uses hostnames in example domain (RFC-6761), so we update local dns resolver.

```bash
LB_IP=$(kubectl get svc -n istio-system -l istio=ingressgateway -o jsonpath='{.items[0]..status.loadBalancer.ingress[0].ip}')
echo "$LB_IP auth.example.com org1.example.com org2.example.com" | sudo tee -a /etc/hosts
```

1. Create org1 and org2 Organziations

```bash
kubectl apply -f organizations.yaml
```

1. Create VS for these two organizations
Note these VirtualServices are very simple, they just hold the hostname and gateway reference. Routes are stored in delegated VS created earilier.
It allows us to keep configuration clean and DRY.

```bash
kubectl apply -f orgs-vs.yaml
```

1. Create user

* In dex, create one user (dex is shared by both orgs, so any orgnanization hostname will work):

```bash
curl -X POST -H 'Content-type: application/json' \
-d '{"email": "[email protected]","password": "mypassword","displayName": "John Doe"}' \
-H 'Authorization: Bearer YWRtaW46Ym9vdHN0cmFwOkdkY05hczEyMw' -k https://org1.example.com/api/v1/auth/users
```

Note the authentication Id returned by the API. Use it in the following 2 requests:

* Map dex user to `admin` user in both organizations:

```bash
curl -X PATCH -k https://org1.example.com/api/v1/entities/users/admin -H "Authorization: Bearer YWRtaW46Ym9vdHN0cmFwOkdkY05hczEyMw" \
-H "Content-Type: application/vnd.gooddata.api+json" -d '{
"data": {
"id": "admin",
"type": "user",
"attributes": {
"authenticationId": "<<authenticationId-returned-above>>",
"email": "[email protected]",
"firstname": "John",
"lastname": "Doe"
}
}
}'
curl -X PATCH -k https://org2.example.com/api/v1/entities/users/admin -H "Authorization: Bearer YWRtaW46Ym9vdHN0cmFwOkdkY05hczEyMw" \
-H "Content-Type: application/vnd.gooddata.api+json" -d '{
"data": {
"id": "admin",
"type": "user",
"attributes": {
"authenticationId": "<<authenticationId-returned-above>>",
"email": "[email protected]",
"firstname": "John",
"lastname": "Doe"
}
}
}'

```

1. Login to UI with username `[email protected]` and password `mypassword` to https://org1.example.com/ or https://org2.example.com/.

## Issues to solve

* ~~job pods keep running, the main container is "Completed" but "istio-proxy" container remains running.~~
RESOLVED by using native sidecars.

* Readiness probe of calcique and afm-exec-api fail for a long time, becuase they are unable to resolve headless service to metadata-api or calcique, respectively.
They will recover enventually, but it takes a few minutes.

* mTLS setup
* ~~TLS setup on Gateway~~ (DONE)
* cert-manager integration
* How to handle organization hostnames that do not match `*.exmaple.com` wildcard?

## Useful links

* [Istio docs](https://istio.io/latest/docs/concepts/traffic-management/)
* [Envoy response flags](https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#config-access-log-format-response-flags)
* [GoodData Docs](https://www.gooddata.com/docs/cloud-native/3.17/install/install-locally/prepare/)
* [Istio by Example (cool!)](https://istiobyexample.dev/)
* [Istio config validator](https://github.com/getyourguide/istio-config-validator)
26 changes: 26 additions & 0 deletions gateway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
name: gooddata-cn-gw
namespace: gooddata
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- "*.example.com"
port:
name: http
number: 80
protocol: HTTP
tls:
httpsRedirect: true
- hosts:
- "*.example.com"
port:
name: https
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: star.example.com
36 changes: 36 additions & 0 deletions gdcn-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
deployVisualExporter: false
license:
existingSecret: gdcn-license

postgresql-ha:
pgpool:
replicaCount: 1
postgresql:
replicaCount: 1
service:
portName: tcp-postgresql

etcd:
service:
clientPortNameOverride: tcp-client
peerPortNameOverride: tcp-peer
metricsPortNameOverride: http-metrics

redis-ha:
hardAntiAffinity: false

image:
pullPolicy: IfNotPresent

# ingress:
# lbProtocol: https

replicaCount: 1

dex:
ingress:
authHost: auth.example.com

metadataApi:
encryptor:
enabled: false
19 changes: 19 additions & 0 deletions istio-virtual-service-dex.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
name: gooddata-cn-dex
namespace: gooddata
spec:
hosts:
- auth.example.com
gateways:
- gooddata-cn-gw
http:
- match:
- uri:
prefix: /dex
route:
- destination:
host: gooddata-cn-dex.gooddata.svc.cluster.local
port:
number: 32000
Loading

0 comments on commit ae54c08

Please sign in to comment.