Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(template): added apache superset (unofficial) with postgres #4891

Draft
wants to merge 6 commits into
base: next
Choose a base branch
from

Conversation

nktnet1
Copy link
Contributor

@nktnet1 nktnet1 commented Jan 21, 2025

Changes

  • added AirBnB's Apache/Superset (unofficial) template

Apache Superset is a popular "Data Visualization and Data Exploration Platform". Currently, it has ~64k stars on GitHub.

Unfortunately, superset does not officially support docker-compose in production:


This template uses an unofficial docker image to allow for production docker deployment. However, this in itself is also quite popular, with nearly 1k stars on GitHub:

and over 5 million downloads on DockerHub:


There are multiple flavours of superset (unofficial) that are supported, e.g.

  • celery
  • postgresql (I chose this one)
  • mysql
  • sqlite (probably not prod)

However, given that it's "unofficial", I wanted to run this through the Coolify team before proceeding with the superset-unofficial-with-celery and superset-unofficial-with-mysql templates, in case the policy is to not accept unofficial images.


NOTE: for the first launch, you will need to enter the superset container using Coolify's terminal feature to run

superset-init

to register the admin user and database. Alternatively, the superset-demo command will do the same, and additionally load demo database/charts/dashboards into the superset instance.

I'm happy to contribute this to documentations-coolify if it gets the green light.


Demonstration:

demo.webm

@nktnet1 nktnet1 changed the title feat(template): added superset unofficial with postgres feat(template): added apache superset (unofficial) with postgres Jan 21, 2025
@peaklabs-dev peaklabs-dev added the ⚙️ Service Issues requesting or PRs adding/fixing service templates. label Jan 21, 2025
@nktnet1
Copy link
Contributor Author

nktnet1 commented Jan 22, 2025

Actually, a potential issue with the "Connect to Predefined Networks" checkbox (necessary to pull data from other databases on Coolify):

Since superset has its own redis cache, the file volume mount superset_config.py will default to Coolify's redis.


For example,

$ docker ps --filter "name=redis" --format "{{.ID}} | {{.Names}}"
bd2e2aedb5bd | redis-qo0gsoowco804scgocwgw88o               # this is superset's redis
3b3169c7f2a9 | coolify-redis

Inspecting superset redis:

$ docker inspect bd2e2aedb5bd | grep DNSNames -A3
                    # Under NetworkSettings -> Networks -> Coolify
                    "DNSNames": [
                        "redis-qo0gsoowco804scgocwgw88o",
                        "bd2e2aedb5bd"
                    ]
                    
                   # Under NetworkSettings -> Networks -> qo0gsoowco804scgocwgw88o
                   # (UUID-generated network from "Connect to Predefined Networks")
                    "DNSNames": [
                        "redis-qo0gsoowco804scgocwgw88o",
                        "redis",
                        "bd2e2aedb5bd"

and inspecting coolify redis:

$ docker inspect 3b3169c7f2a9 | grep DNSNames -A3
                   # Under NetworkSettings -> Networks -> Coolify
                    "DNSNames": [
                        "coolify-redis",
                        "redis",
                        "3b3169c7f2a9"

Since the superset container's superset_config.py has

"CACHE_REDIS_URL": f"redis://:{os.getenv('REDIS_PASSWORD')}@redis:6379/1",

and the superset container is in both networks, I assume the host redis now refers to Coolify's redis after the "Connect to Predefined Networks" is enabled (given that coolify-networks is injected at the beginning first and takes precedence).

Is there a way around this?
Basically need a way to get the redis-qo0gsoowco804scgocwgw88o host into the python file.
Would be great if the service's generated UUID can be exposed as an environment variable, e.g. ${COOLIFY_SERVICE_UUID}, and available to all containers.

Or perhaps, if possible, move the priority so that the internal network takes precedence over Coolify's network.


Worst case, the redis service/host could also just be renamed to something like `superset_unofficial_with_postgres_redis"

@nktnet1
Copy link
Contributor Author

nktnet1 commented Jan 22, 2025

Found a workaround (b617a08) by using COOLIFY_CONTAINER_NAME:

By adding the following python code:

COOLIFY_UUID = os.getenv('COOLIFY_CONTAINER_NAME').strip('"').split('-')[-1]

# ...

# Then, for example with postgres db, we can use our new db-{COOLIFY_UUID} host:
SQLALCHEMY_DATABASE_URI = f"postgresql+psycopg2://{os.getenv('POSTGRES_USER')}:{os.getenv('POSTGRES_PASSWORD')}@db-{COOLIFY_UUID}:5432/{os.getenv('POSTGRES_DB')}"

COOLIFY_CONTAINER_NAME double quotes

An issue I discovered was that the environment variable COOLIFY_CONTAINER_NAME is excessively surrounded in double quotes:

double-quotes

Hence why the .strip('"') is necessary.

This likely stems from here:

if ($this->application->environment_variables_preview->where('key', 'COOLIFY_CONTAINER_NAME')->isEmpty()) {
$envs->push("COOLIFY_CONTAINER_NAME=\"{$this->container_name}\"");
}

and/or here:

if ($this->application->environment_variables->where('key', 'COOLIFY_CONTAINER_NAME')->isEmpty()) {
$envs->push("COOLIFY_CONTAINER_NAME=\"{$this->container_name}\"");
}


Expose COOLIFY_UUID

As per the original request in #2545, this would simplify the need to extract the UUID from the container name for use as a network host.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⚙️ Service Issues requesting or PRs adding/fixing service templates.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants