Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/container image publish #226

Open
wants to merge 30 commits into
base: main
Choose a base branch
from

Conversation

BWagener
Copy link
Contributor

@BWagener BWagener commented Feb 2, 2025

This PR adds a workflow to build and publish a container image of UltraSinger.

The image currently has a size of ~12Gi.
dive output:

┃ ● Layers ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Permission     UID:GID       Size  Filetree
Cmp   Size  Command                                                                                                           -rw-r--r--         0:0      17 kB  ├── NGC-DL-CONTAINER-LICENSE
     78 MB  FROM 2573e0d8158209e                                                                                              drwxr-xr-x         0:0     8.4 GB  ├── app
     11 MB  RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update && apt-get install -y --no-install-recommends     gnupg2 curl c drwxr-xr-x   1000:1000     8.4 GB  │   └── UltraSinger
    162 MB  RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update && apt-get install -y --no-install-recommends     cuda-cudart-1 drwxr-xr-x   1000:1000     8.4 GB  │       ├── .venv
      46 B  RUN |1 TARGETARCH=amd64 /bin/sh -c echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf     && echo "/us drwxr-xr-x   1000:1000      51 kB  │       │   ├── bin
     17 kB  COPY NGC-DL-CONTAINER-LICENSE / # buildkit                                                                        -rw-r--r--   1000:1000     9.0 kB  │       │   │   ├── Activate.ps1
    2.1 GB  RUN |1 TARGETARCH=amd64 /bin/sh -c apt-get update && apt-get install -y --no-install-recommends     cuda-librarie -rw-r--r--   1000:1000     2.0 kB  │       │   │   ├── activate
    263 kB  RUN |1 TARGETARCH=amd64 /bin/sh -c apt-mark hold ${NV_LIBCUBLAS_PACKAGE_NAME} ${NV_LIBNCCL_PACKAGE_NAME} # buildk -rw-r--r--   1000:1000      919 B  │       │   │   ├── activate.csh
    3.1 kB  COPY entrypoint.d/ /opt/nvidia/entrypoint.d/ # buildkit                                                           -rw-r--r--   1000:1000     2.2 kB  │       │   │   ├── activate.fish
    2.5 kB  COPY nvidia_entrypoint.sh /opt/nvidia/ # buildkit                                                                 -rwxr-xr-x   1000:1000      231 B  │       │   │   ├── alembic
    838 MB  apt-get update     && apt-get install git python3-pip python3.10-venv ffmpeg -y      && apt-get clean      && rm  -rwxr-xr-x   1000:1000      238 B  │       │   │   ├── black
     12 kB  #(nop) COPY file:89ab23ea6981b9bc7628756191ded3af46b8ee7138e3f2975720dd346e344f0e in /app/UltraSinger/requirement -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── blackd
     12 kB  chown -R 1000:1000 /app/UltraSinger                                                                               -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── chardetect
     16 MB  python3 -m venv $VIRTUAL_ENV                                                                                      -rwxr-xr-x   1000:1000      232 B  │       │   │   ├── coloredlogs
    8.4 GB  pip install --no-cache-dir -r requirements.txt     && pip install --no-cache-dir torch==2.3.0 torchvision==0.18.0 -rwxr-xr-x   1000:1000      270 B  │       │   │   ├── convert-caffe2-to-onnx
    360 kB  #(nop) COPY dir:237d48bd29a84c007afa32dcb1900d39b95dbf972707b3954bf00bea846d36df in /app/UltraSinger              -rwxr-xr-x   1000:1000      270 B  │       │   │   ├── convert-onnx-to-caffe2
                                                                                                                              -rwxr-xr-x   1000:1000      969 B  │       │   │   ├── crepe
                                                                                                                              -rwxr-xr-x   1000:1000      247 B  │       │   │   ├── ct2-fairseq-converter
                                                                                                                              -rwxr-xr-x   1000:1000      246 B  │       │   │   ├── ct2-marian-converter
                                                                                                                              -rwxr-xr-x   1000:1000      251 B  │       │   │   ├── ct2-openai-gpt2-converter
│ Layer Details ├──────────────────────────────────────────────────────────────────────────────────────────────────────────── -rwxr-xr-x   1000:1000      250 B  │       │   │   ├── ct2-opennmt-py-converter
                                                                                                                              -rwxr-xr-x   1000:1000      250 B  │       │   │   ├── ct2-opennmt-tf-converter
Tags:   (unavailable)                                                                                                         -rwxr-xr-x   1000:1000      247 B  │       │   │   ├── ct2-opus-mt-converter
Id:     c9dd5093afd6deeadc5e82a54e2d624996f89b0d9572ae5671a4118e76b6a23f.tar                                                  -rwxr-xr-x   1000:1000      252 B  │       │   │   ├── ct2-transformers-converter
Digest: sha256:c9dd5093afd6deeadc5e82a54e2d624996f89b0d9572ae5671a4118e76b6a23f                                               -rwxr-xr-x   1000:1000      971 B  │       │   │   ├── demucs
Command:                                                                                                                      -rwxr-xr-x   1000:1000      230 B  │       │   │   ├── dora
pip install --no-cache-dir -r requirements.txt     && pip install --no-cache-dir torch==2.3.0 torchvision==0.18.0 torchaudio= -rwxr-xr-x   1000:1000      234 B  │       │   │   ├── f2py
=2.3.0 --index-url https://download.pytorch.org/whl/cu121     && pip install --no-cache-dir tensorflow[and-cuda]==2.16.1      -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── fabric
                                                                                                                              -rwxr-xr-x   1000:1000      235 B  │       │   │   ├── fonttools
                                                                                                                              -rwxr-xr-x   1000:1000      233 B  │       │   │   ├── futurize
                                                                                                                              -rwxr-xr-x   1000:1000     2.5 kB  │       │   │   ├── get_gprof
                                                                                                                              -rwxr-xr-x   1000:1000     1.7 kB  │       │   │   ├── get_objgraph
                                                                                                                              -rwxr-xr-x   1000:1000      251 B  │       │   │   ├── google-oauthlib-tool
                                                                                                                              -rwxr-xr-x   1000:1000      257 B  │       │   │   ├── huggingface-cli
                                                                                                                              -rwxr-xr-x   1000:1000      234 B  │       │   │   ├── humanfriendly
                                                                                                                              -rwxr-xr-x   1000:1000      259 B  │       │   │   ├── imageio_download_bin
                                                                                                                              -rwxr-xr-x   1000:1000      255 B  │       │   │   ├── imageio_remove_bin
                                                                                                                              -rwxr-xr-x   1000:1000      265 B  │       │   │   ├── import_pb_to_tensorboard
                                                                                                                              -rwxr-xr-x   1000:1000      227 B  │       │   │   ├── isort
                                                                                                                              -rwxr-xr-x   1000:1000      261 B  │       │   │   ├── isort-identify-imports
                                                                                                                              -rwxr-xr-x   1000:1000      223 B  │       │   │   ├── isympy
│ Image Details ├──────────────────────────────────────────────────────────────────────────────────────────────────────────── -rwxr-xr-x   1000:1000      253 B  │       │   │   ├── lightning
                                                                                                                              -rwxr-xr-x   1000:1000      231 B  │       │   │   ├── mako-render
Image name: ghcr.io/bwagener/ultrasinger:sha-c1b8004                                                                          -rwxr-xr-x   1000:1000      238 B  │       │   │   ├── markdown-it
Total Image size: 12 GB                                                                                                       -rwxr-xr-x   1000:1000      232 B  │       │   │   ├── markdown_py
Potential wasted space: 12 MB                                                                                                 -rwxr-xr-x   1000:1000      242 B  │       │   │   ├── mido-connect
Image efficiency score: 99 %                                                                                                  -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── mido-play
                                                                                                                              -rwxr-xr-x   1000:1000      240 B  │       │   │   ├── mido-ports
Count   Total Space  Path                                                                                                     -rwxr-xr-x   1000:1000      240 B  │       │   │   ├── mido-serve
    3        1.6 MB  /var/cache/debconf/templates.dat                                                                         -rwxr-xr-x   1000:1000      223 B  │       │   │   ├── nltk
    5        1.2 MB  /var/log/dpkg.log                                                                                        -rwxr-xr-x   1000:1000      251 B  │       │   │   ├── normalizer
    6        1.0 MB  /var/lib/dpkg/status                                                                                     -rwxr-xr-x   1000:1000     3.0 kB  │       │   │   ├── num2words
    2        1.0 MB  /var/cache/debconf/templates.dat-old                                                                     -rwxr-xr-x   1000:1000      206 B  │       │   │   ├── numba
    5        944 kB  /var/lib/dpkg/status-old                                                                                 -rwxr-xr-x   1000:1000      251 B  │       │   │   ├── onnxruntime_test
    2        266 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/setuptools/_vendor/more_itertools/more.py            -rwxr-xr-x   1000:1000      227 B  │       │   │   ├── optuna
    2        248 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/setuptools/_vendor/more_itertools/__pycache__/more.c -rwxr-xr-x   1000:1000      235 B  │       │   │   ├── pasteurize
python-310.pyc                                                                                                                -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── pip
    2        235 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/pkg_resources/__init__.py                            -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── pip3
    2        216 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/pkg_resources/__pycache__/__init__.cpython-310.pyc   -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── pip3.10
    2        176 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/setuptools/command/easy_install.py                   -rwxr-xr-x   1000:1000      239 B  │       │   │   ├── py.test
    2        152 kB  /app/UltraSinger/.venv/lib/python3.10/site-packages/setuptools/gui-arm64.exe                             -rwxr-xr-x   1000:1000      252 B  │       │   │   ├── pyannote-audio-eval

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant