Pass values from Secrets to `extra_settings`

Currently, AWX Operator does not ability to inject Secrets to extra_settings.

However, since Secrets are injectable for environment variables, we can indirectly inject values from Secrets into extra_settings by specifying values that refers to the environment variables.

Concept

Since the values for extra_settings will be embedded as Python code into settings.py, we can use any python code as values for extra_settings. So the key concepts are:

Add environment variables for both task and web pod with valueFrom to inject values from Secrets to environment variables.
Specify os.getenv() in values for extra_settings.

Simple examples

This is a simple example to inject values from Secrets to extra_settings.

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  ...

  # Add new environment variables with "valueFrom" to task pod
  task_extra_env: |
    - name: AZUREAD_OAUTH2_KEY
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: key
    - name: AZUREAD_OAUTH2_SECRET
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: secret

  # Completely the same environment variables should be added to web pod
  web_extra_env: |
    - name: AZUREAD_OAUTH2_KEY
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: key
    - name: AZUREAD_OAUTH2_SECRET
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: secret

  # Specify "os.getenv()" that refers environment variables that newly added above
  extra_settings:
    - setting: SOCIAL_AUTH_AZUREAD_OAUTH2_KEY
      value: os.getenv("AZUREAD_OAUTH2_KEY")
    - setting: SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET
      value: os.getenv("AZUREAD_OAUTH2_SECRET")

Complex examples

This is an example to inject values from Secrets as a part of Dictionary. The Dictionary should be passed as multi-line strings.

Important

If we want to pass multi-line strings including os.getenv() as values for extra_settings, indentation have to be noted since the values will be embedded into ConfigMap that templated by Jinja, and ConfigMap has 4 spaces indentation per lines. We must not destroy the indentation of the ConfigMap.

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  ...

  # Add new environment variables with "valueFrom" to task pod
  task_extra_env: |
    - name: SAML_IDP_SIGNING_CERT
      valueFrom:
        secretKeyRef:
          name: awx-saml-idp-signing-cert
          key: x509cert

  # Completely the same environment variables should be added to web pod
  web_extra_env: |
    - name: SAML_IDP_SIGNING_CERT
      valueFrom:
        secretKeyRef:
          name: awx-saml-idp-signing-cert
          key: x509cert

  # Specify "os.getenv()" that refers environment variables that newly added above
  extra_settings:
    - setting: SOCIAL_AUTH_SAML_ENABLED_IDPS 
      # Define value as multi-line strings with 2 spaces indentation (by "|2"),
      # then pass the actual value with 6 spaces indentation
      # to add 4 leading spaces for each lines to keep indentation in ConfigMap correctly
      value: |2
            {
              "saml_ms_adfs": {
                "x509cert": os.getenv("SAML_IDP_SIGNING_CERT"),
                "attr_last_name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname",
                "attr_first_name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname",
                "attr_email": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
                "attr_username": "name_id",
                "attr_user_permanent_id": "name_id",
                "entity_id": "https://adfs.domain.com/adfs/services/trust",
                "url": "https://adfs.domain.com/adfs/ls",
              }
            }

This will generate following ConfigMap. As you can see, to include all lines into under settings, 4 spaces indentation is important.

---
apiVersion: v1
kind: ConfigMap
...
data:
  ...
  settings: |
    import os
    import socket
    ...
    SOCIAL_AUTH_SAML_ENABLED_IDPS =     {
      "saml_ms_adfs": {
        "x509cert": os.getenv("SAML_IDP_SIGNING_CERT"),
        "attr_last_name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname",
        "attr_first_name": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname",
        "attr_email": "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
        "attr_username": "name_id",
        "attr_user_permanent_id": "name_id",
        "entity_id": "https://adfs.domain.com/adfs/services/trust",
        "url": "https://adfs.domain.com/adfs/ls",
      }
    }

Tip

If you don't want to care about indentation, alternatively it is also possible to pass a Dictionary on a single line, as long as it follows Python syntax.

Tip

The above example assumes that the certificate is stored in secret as a single line of text data, not including line breaks, headers, or footers.

kubectl -n awx create secret generic awx-saml-idp-signing-cert \
  --from-literal=x509cert=MIIDZTCC........6xGPxz26

If your secrets contains whole the certificate, you can use replace() after getenv(), to remove line breaks, headers, and footers.

"x509cert": os.getenv("SAML_IDP_SIGNING_CERT").replace("\n", "").replace("-----BEGIN CERTIFICATE-----", "").replace("-----END CERTIFICATE----", ""),

Appendix: Add extra settings by mounting settings files instead of using `extra_settings`

If you have a lot of settings to add, or want to pass dictionaries or lists, it might be hard to maintain them in extra_settings.

In that case, you can create settings files as pure Python code and mount them under /etc/tower/conf.d directory on the task and web pods, since AWX reads any *.py files in /etc/tower/conf.d directory as settings files on its start up.

Add extra settings by mounting ConfigMap

This is an example extra_settings.py file that contain multiple extra settings.

AD_HOC_COMMANDS = [
    "command",
    "shell",
    "yum",
    "apt",
    "apt_key",
    "apt_repository",
    "apt_rpm",
    "service",
    "group",
    "user",
    "mount",
    "ping",
    "selinux",
    "setup",
    "win_ping",
    "win_service",
    "win_updates",
    "win_group",
    "win_user",
]
AWX_ISOLATION_SHOW_PATHS = [
    "/etc/pki/ca-trust:/etc/pki/ca-trust:O",
    "/usr/share/pki:/usr/share/pki:O",
]
AWX_TASK_ENV = {
    "HTTPS_PROXY": "http://proxy.example.com:3128",
    "HTTP_PROXY": "http://proxy.example.com:3128",
    "NO_PROXY": "127.0.0.1,localhost,.example.com",
}
GALAXY_TASK_ENV = {
    "ANSIBLE_FORCE_COLOR": "false",
    "GIT_SSH_COMMAND": "ssh -o StrictHostKeyChecking=no"
}

Create a ConfigMap with above extra_settings.py file.

kubectl -n awx create configmap awx-extra-settings-configmap --from-file=extra_settings.py

Then, mount the ConfigMap to the task and web pods.

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  ...

  extra_volumes: |
    - name: awx-extra-settings
      configMap:
        name: awx-extra-settings-configmap
        items:
          - key: extra_settings.py
            path: extra_settings.py

  task_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

  web_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

Refer sensitive values from Secrets from the settings files

If you want to embed the values from Secrets into the settings files, you can use os.getenv() in extra_settings.py as well.

import os

...
SOCIAL_AUTH_AZUREAD_OAUTH2_KEY = os.getenv("AZUREAD_OAUTH2_KEY")
SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET = os.getenv("AZUREAD_OAUTH2_SECRET")
SOCIAL_AUTH_SAML_ENABLED_IDPS = {
    "saml_ms_adfs": {
        "x509cert": os.getenv("SAML_IDP_SIGNING_CERT"),
        ...
    }
}

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  ...

  task_extra_env: |
    - name: AZUREAD_OAUTH2_KEY
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: key
    - name: AZUREAD_OAUTH2_SECRET
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: secret
    - name: SAML_IDP_SIGNING_CERT
      valueFrom:
        secretKeyRef:
          name: awx-saml-idp-signing-cert
          key: x509cert

  web_extra_env: |
    - name: AZUREAD_OAUTH2_KEY
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: key
    - name: AZUREAD_OAUTH2_SECRET
      valueFrom:
        secretKeyRef:
          name: awx-azuread-oauth2-secret
          key: secret
    - name: SAML_IDP_SIGNING_CERT
      valueFrom:
        secretKeyRef:
          name: awx-saml-idp-signing-cert
          key: x509cert

  extra_volumes: |
    - name: awx-extra-settings
      configMap:
        name: awx-extra-settings-configmap
        items:
          - key: extra_settings.py
            path: extra_settings.py

  task_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

  web_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

Embed sensitive values directly into the settings files

Alternatively, if you want to embed sensitive values into the settings files directly, you can store extra_settings.py as a Secret instead of a ConfigMap.

import os

...
SOCIAL_AUTH_AZUREAD_OAUTH2_KEY = "****************"
SOCIAL_AUTH_AZUREAD_OAUTH2_SECRET = "****************"
SOCIAL_AUTH_SAML_ENABLED_IDPS = {
    "saml_ms_adfs": {
        "x509cert": "MIIDZTCC........6xGPxz26",
        ...
    }
}

kubectl -n awx create secret generic awx-extra-settings-secret --from-file=extra_settings.py

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  ...

  extra_volumes: |
    - name: awx-extra-settings
      secret:
        secretName: awx-extra-settings-secret
        items:
          - key: extra_settings.py
            path: extra_settings.py

  task_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

  web_extra_volume_mounts: |
    - name: awx-extra-settings
      mountPath: /etc/tower/conf.d/extra_settings.py
      subPath: extra_settings.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

extra-settings.md

extra-settings.md

Pass values from Secrets to `extra_settings`

Table of Contents

Concept

Simple examples

Complex examples

Appendix: Add extra settings by mounting settings files instead of using `extra_settings`

Add extra settings by mounting ConfigMap

Refer sensitive values from Secrets from the settings files

Embed sensitive values directly into the settings files

Files

extra-settings.md

Latest commit

History

extra-settings.md

File metadata and controls

Pass values from Secrets to extra_settings

Table of Contents

Concept

Simple examples

Complex examples

Appendix: Add extra settings by mounting settings files instead of using extra_settings

Add extra settings by mounting ConfigMap

Refer sensitive values from Secrets from the settings files

Embed sensitive values directly into the settings files

Pass values from Secrets to `extra_settings`

Appendix: Add extra settings by mounting settings files instead of using `extra_settings`