Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Salt fm updates #11

Merged
merged 13 commits into from
Dec 6, 2024
23 changes: 14 additions & 9 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
name: ci
# See https://docs.docker.com/build/ci/github-actions/examples

permissions:
contents: read
packages: write

env:
DOCKER_BUILDKIT: 1

Expand All @@ -21,7 +26,7 @@ jobs:
-
name: Docker meta
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
# list of Docker images to use as base name for tags
images: |
Expand All @@ -39,29 +44,29 @@ jobs:
type=schedule
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
submodules: recursive
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
-
name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
-
name: Login to GitHub Container Registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build and push
uses: docker/build-push-action@v3
uses: docker/build-push-action@v6
with:
context: .
file: ./Dockerfile
Expand All @@ -74,7 +79,7 @@ jobs:
-
name: Update repo description
if: github.event_name != 'pull_request'
uses: peter-evans/dockerhub-description@v2
uses: peter-evans/dockerhub-description@v4
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
Expand Down
47 changes: 30 additions & 17 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# syntax=docker/dockerfile:1.4
# Stage 1. Check out LLVM source code and run the build.
FROM launcher.gcr.io/google/debian11:latest as builder
FROM debian:12 as builder
LABEL maintainer "ParaTools Inc."

# Install build dependencies of llvm.
Expand All @@ -20,12 +20,11 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked --mount=type=cache,t
apt-get update
apt-get install -y --no-install-recommends ca-certificates \
build-essential cmake ccache make python3 zlib1g wget unzip git
cmake --version
EOC

# use ccache (make it appear in path earlier then /usr/bin/gcc etc)
RUN for p in gcc g++ clang clang++ cc c++; do ln -vs /usr/bin/ccache /usr/local/bin/$p; done

ARG CI=false
ARG LLVM_VER=18
# Clone LLVM repo. A shallow clone is faster, but pulling a cached repository is faster yet
RUN --mount=type=cache,target=/git <<EOC
echo "Checking out LLVM."
Expand All @@ -41,7 +40,7 @@ RUN --mount=type=cache,target=/git <<EOC
echo "Running under CI. \$CI=$CI. Shallow cloning will be used if a clone is required."
# export SHALLOW='--depth=1'
fi
if mkdir llvm-project && git --git-dir=/git/llvm-project.git -C llvm-project pull origin release/15.x --ff-only
if mkdir llvm-project && git --git-dir=/git/llvm-project.git -C llvm-project pull origin release/${LLVM_VER}.x --ff-only
then
echo "WARNING: Using cached llvm git repository and pulling updates"
cp -r /git/llvm-project.git /llvm-project/.git
Expand All @@ -50,7 +49,7 @@ RUN --mount=type=cache,target=/git <<EOC
echo "Cloning a fresh LLVM repository"
git clone --separate-git-dir=/git/llvm-project.git \
${SHALLOW:-} --single-branch \
--branch=release/15.x \
--branch=release/${LLVM_VER}.x \
--filter=blob:none \
https://github.com/llvm/llvm-project.git
if [ -f /llvm-project/.git ]; then
Expand Down Expand Up @@ -92,23 +91,35 @@ RUN --mount=type=cache,target=/ccache/ <<EOC
exit 1
fi
ccache -s
#Configure the build
if uname -a | grep x86 ; then export LLVM_TARGETS_TO_BUILD="-DLLVM_TARGETS_TO_BUILD=X86"; fi
cmake -GNinja \
-DCMAKE_INSTALL_PREFIX=/tmp/llvm \
-DCMAKE_MAKE_PROGRAM=/usr/local/bin/ninja \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" \
-DLLVM_CCACHE_BUILD=On \
-DLLVM_ENABLE_PROJECTS="flang;clang;clang-tools-extra;mlir;openmp" \
-DLLVM_ENABLE_RUNTIMES="compiler-rt" \
${LLVM_TARGETS_TO_BUILD} \
-S /llvm-project/llvm -B /llvm-project/llvm/build

# Build libraries, headers, and binaries
# Do build
ccache -s
cd /llvm-project/llvm/build
# Actually do the build
ninja install-llvm-libraries install-llvm-headers \
# Actually do the build on nproc - 1 cores unless nproc == 2
ninja -j $(( $(nproc --ignore=4) > 2 ? $(nproc --ignore=1) : 2)) \
install-llvm-libraries install-llvm-headers install-llvm-config install-cmake-exports \
install-clang-libraries install-clang-headers install-clang install-clang-cmake-exports \
install-clang-resource-headers install-llvm-config install-cmake-exports
ccache -s
install-clang-resource-headers \
install-mlir-headers install-mlir-libraries install-mlir-cmake-exports \
install-openmp-resource-headers \
install-compiler-rt \
install-flang-libraries install-flang-headers install-flang-new install-flang-cmake-exports \
install-flangFrontend install-flangFrontendTool
git -C /llvm-project status
cp -r /tmp/llvm /usr/local/
ccache -s
EOC

# Patch installed cmake exports/config files to not throw an error if not all components are installed
Expand All @@ -121,7 +132,7 @@ RUN <<EOC
EOC

# Stage 2. Produce a minimal release image with build results.
FROM launcher.gcr.io/google/debian11:latest
FROM debian:12
LABEL maintainer "ParaTools Inc."

# Install packages for minimal useful image.
Expand All @@ -132,7 +143,8 @@ EOC

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked --mount=type=cache,target=/var/lib/apt,sharing=locked <<EOC
apt-get update
apt-get install -y --no-install-recommends libstdc++-10-dev \
# libstdc++-10-dev \
apt-get install -y --no-install-recommends \
ccache libz-dev libelf1 libtinfo-dev make binutils cmake git \
gcc g++ wget
rm -rf /var/lib/apt/lists/*
Expand Down Expand Up @@ -163,11 +175,12 @@ RUN --mount=type=cache,target=/home/salt/ccache <<EOC
echo "verbose=off" > ~/.wgetrc
wget http://tau.uoregon.edu/tau.tgz || wget http://fs.paratools.com/tau-mirror/tau.tgz
tar xzvf tau.tgz
# GIT_SSL_NO_VERIFY=true git clone --recursive --depth=1 --single-branch https://github.com/UO-OACISS/tau2.git
cd tau*
./installtau -prefix=/usr/local/ -cc=gcc -c++=g++\
-bfd=download -unwind=download -dwarf=download -otf=download -pthread -iowrapper -j
./installtau -prefix=/usr/local/ -cc=clang -c++=clang++\
-bfd=download -unwind=download -dwarf=download -otf=download -pthread -iowrapper -j
./installtau -prefix=/usr/local/ -cc=gcc -c++=g++ -fortran=gfortran\
-bfd=download -unwind=download -dwarf=download -otf=download -zlib=download -pthread -j
./installtau -prefix=/usr/local/ -cc=clang -c++=clang++ -fortran=flang\
-bfd=download -unwind=download -dwarf=download -otf=download -zlib=download -pthread -j
cd ..
rm -rf tau*
ccache -s
Expand Down
2 changes: 0 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,5 @@ Using the official docker GitHub Actions with their support for caching maximize
efficiency of building the image(s) during CI.
Very useful examples are available [here](https://docs.docker.com/build/ci/github-actions/examples).



[SALT]: https://github.com/ParaToolsInc/salt
[git submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
2 changes: 1 addition & 1 deletion patches
Loading