-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: upload contracts to CF R2 from main #100
Changes from all commits
dbcafbd
3a6fd1c
30dfc37
49c07dd
cc6a1b5
62b2b58
8700b9c
50db718
26d65db
6c9d0a2
a46fa3d
98d3fa8
9301582
3d6844b
8240087
a1e8954
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,66 @@ | ||
# Workflow to fetch the latest commit hash on the main branch and upload artifacts to CF storage. | ||
name: Build and upload from main | ||
on: | ||
workflow_dispatch: | ||
|
||
concurrency: ${{ github.workflow }}-${{ github.ref }} | ||
|
||
jobs: | ||
stellar-contract-names: | ||
name: Get all Stellar contracts | ||
runs-on: blacksmith-2vcpu-ubuntu-2204 | ||
outputs: | ||
releases: ${{ steps.prepare-release.outputs.releases }} | ||
commit_hash: ${{ steps.get-commit-hash.outputs.hash }} | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Install dependencies | ||
run: sudo apt-get install -y jq | ||
|
||
- name: Get latest commit hash | ||
id: get-commit-hash | ||
run: echo "hash=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT" | ||
|
||
- name: Prepare JSON output for release | ||
id: prepare-release | ||
run: | | ||
RELEASES_JSON=$(find contracts -maxdepth 1 -mindepth 1 -type d | sed 's|contracts/||' | jq -R . | jq -s --arg commit "${{ steps.get-commit-hash.outputs.hash }}" 'map({ | ||
package_name: ., | ||
version: $commit, | ||
package_git_tag: "\(.)_\($commit)" | ||
})') | ||
Comment on lines
+30
to
+34
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is overkill, the commit is the same now, we just need to list all contract names There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should also consider the existing There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But we aren't releasing to crates.io here. Take a look at how the |
||
echo "releases=$(echo "$RELEASES_JSON" | jq -c)" >> "$GITHUB_OUTPUT" | ||
|
||
build: | ||
needs: stellar-contract-names | ||
uses: ./.github/workflows/reusable-build.yaml | ||
with: | ||
commit-hash: ${{ needs.stellar-contract-names.outputs.commit_hash }} | ||
|
||
upload: | ||
needs: [stellar-contract-names, build] | ||
strategy: | ||
matrix: | ||
releases: ${{ fromJson(needs.stellar-contract-names.outputs.releases) }} | ||
|
||
uses: ./.github/workflows/reusable-upload.yaml | ||
permissions: | ||
id-token: write | ||
contents: read | ||
with: | ||
package-name: ${{ matrix.releases.package_name }} | ||
package-version: ${{ matrix.releases.version }} | ||
package-git-tag: ${{ matrix.releases.package_git_tag }} | ||
artifact-name: ${{ needs.build.outputs.artifact-name }} | ||
artifact-path: ${{ needs.build.outputs.artifact-path }} | ||
cf-bucket-name: ${{ vars.CF_BUCKET_NAME }} | ||
cf-config-bucket-root-key: ${{ vars.CF_BUCKET_ROOT_KEY }} | ||
github-release: false | ||
secrets: | ||
github-token: ${{ secrets.PAT_TOKEN }} | ||
cf-endpoint-url: ${{ secrets.CF_ENDPOINT_URL }} | ||
cf-bucket-access-key-id: ${{ secrets.CF_BUCKET_ACCESS_KEY_ID }} | ||
cf-bucket-secret-access-key: ${{ secrets.CF_BUCKET_SECRET_ACCESS_KEY }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,15 +6,14 @@ on: | |
pull_request: | ||
branches: | ||
- main | ||
- 'releases/**' | ||
- "releases/**" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just |
||
types: [closed] | ||
|
||
workflow_dispatch: | ||
|
||
concurrency: ${{ github.workflow }}-${{ github.ref }} | ||
|
||
jobs: | ||
|
||
# Publishes a release in case the release isn't published | ||
publish-release: | ||
name: Publish releases | ||
|
@@ -24,21 +23,25 @@ jobs: | |
((github.event.pull_request.merged == true) && | ||
contains(github.event.pull_request.labels.*.name, 'release')) | ||
|
||
runs-on: blacksmith-8vcpu-ubuntu-2204 | ||
runs-on: blacksmith-2vcpu-ubuntu-2204 | ||
|
||
outputs: | ||
releases: ${{ steps.prepare-matrix.outputs.releases }} | ||
commit_hash: ${{ steps.get-commit-hash.outputs.hash }} | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
token: ${{ secrets.PAT_TOKEN }} | ||
|
||
- name: Install Rust toolchain | ||
uses: dtolnay/rust-toolchain@stable | ||
|
||
- name: Get commit hash | ||
id: get-commit-hash | ||
run: echo "hash=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" | ||
|
||
# Creates git tags and publishes the crates of the new releases | ||
- name: Publish release | ||
id: publish-release | ||
|
@@ -56,35 +59,33 @@ jobs: | |
run: | | ||
echo "releases=$(echo '${{ steps.publish-release.outputs.releases }}' | jq -c '.')" >> $GITHUB_OUTPUT | ||
|
||
# Creates other artifacts needed (`wasm` files) | ||
build-and-upload: | ||
name: Build artifacts for ${{ matrix.releases.package_name }}-v${{ matrix.releases.version }} | ||
build: | ||
needs: publish-release | ||
uses: ./.github/workflows/reusable-build.yaml | ||
with: | ||
commit-hash: ${{ needs.publish-release.outputs.commit_hash }} | ||
Comment on lines
+64
to
+66
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this correct? This is creating a build on the current commit hash instead of using the release tags created for every contract by release-plz. So, the built wasm files won't correspond to the release tag? The release matrix needs to be applied for both build and upload, unless all release tags are the same commit |
||
|
||
# Once a release is done for a package, we iterate on each of these packages and build its corresponding artifacts and upload them | ||
upload: | ||
needs: [publish-release, build] | ||
strategy: | ||
matrix: | ||
releases: ${{ fromJson(needs.publish-release.outputs.releases) }} | ||
|
||
uses: ./.github/workflows/reusable-build-upload.yaml | ||
|
||
uses: ./.github/workflows/reusable-upload.yaml | ||
permissions: | ||
id-token: write | ||
contents: read | ||
|
||
with: | ||
package-name: "${{ matrix.releases.package_name }}" | ||
package-version: "${{ matrix.releases.version }}" | ||
package-git-tag: "${{ matrix.releases.tag }}" | ||
|
||
# CF Bucket related variables | ||
cf-bucket-name: "${{ vars.CF_BUCKET_NAME }}" | ||
|
||
# The root key to be used for accessing the configs. (ex: `test-root-key` puts releases in `test-root-key/*`) | ||
cf-config-bucket-root-key: "${{ vars.CF_BUCKET_ROOT_KEY }}" | ||
|
||
package-name: ${{ matrix.releases.package_name }} | ||
package-version: ${{ matrix.releases.version }} | ||
package-git-tag: ${{ matrix.releases.tag }} | ||
artifact-name: ${{ needs.build.outputs.artifact-name }} | ||
artifact-path: ${{ needs.build.outputs.artifact-path }} | ||
cf-bucket-name: ${{ vars.CF_BUCKET_NAME }} | ||
cf-config-bucket-root-key: ${{ vars.CF_BUCKET_ROOT_KEY }} | ||
github-release: true | ||
secrets: | ||
github-token: "${{ secrets.PAT_TOKEN }}" | ||
cf-endpoint-url: "${{ secrets.CF_ENDPOINT_URL }}" | ||
github-token: ${{ secrets.PAT_TOKEN }} | ||
cf-endpoint-url: ${{ secrets.CF_ENDPOINT_URL }} | ||
cf-bucket-access-key-id: ${{ secrets.CF_BUCKET_ACCESS_KEY_ID }} | ||
cf-bucket-secret-access-key: ${{ secrets.CF_BUCKET_SECRET_ACCESS_KEY }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,67 @@ | ||
name: "Build Contracts" | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
commit-hash: | ||
description: "The commit hash to build from" | ||
type: string | ||
required: true | ||
outputs: | ||
artifact-name: | ||
description: "Name of the uploaded artifact containing all builds" | ||
value: ${{ jobs.build.outputs.artifact-name }} | ||
artifact-path: | ||
description: "Path of the uploaded artifact containing all builds" | ||
value: ${{ jobs.build.outputs.artifact-path }} | ||
|
||
jobs: | ||
build: | ||
runs-on: blacksmith-8vcpu-ubuntu-2204 | ||
outputs: | ||
artifact-name: ${{ steps.set-artifact-name.outputs.name }} | ||
artifact-path: ${{ steps.set-artifact-name.outputs.path }} | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
|
||
- name: Checkout specific commit | ||
run: git checkout ${{ inputs.commit-hash }} | ||
|
||
- name: Install Rust toolchain | ||
uses: dtolnay/rust-toolchain@stable | ||
with: | ||
toolchain: 1.81.0 | ||
ahramy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
targets: wasm32-unknown-unknown | ||
|
||
- name: Set artifact name | ||
id: set-artifact-name | ||
run: | | ||
echo "name=wasm-builds-${{ inputs.commit-hash }}" >> $GITHUB_OUTPUT | ||
echo "path=target/wasm32-unknown-unknown/release" >> $GITHUB_OUTPUT | ||
|
||
- name: Build all contracts | ||
run: | | ||
# Install Stellar CLI compatible with the soroban-sdk version in Cargo.toml | ||
cargo install --locked stellar-cli --version ^22 --features opt | ||
|
||
# Build all contracts | ||
stellar contract build | ||
./optimize.sh | ||
|
||
# Process in the release directory | ||
cd target/wasm32-unknown-unknown/release | ||
|
||
# Remove unoptimized files and rename optimized ones | ||
# This ensures we only keep the optimized versions | ||
find . -type f -name "*.wasm" ! -name "*.optimized.wasm" -maxdepth 1 -delete | ||
find . -name "*.optimized.wasm" -maxdepth 1 -exec sh -c 'mv "$0" "${0%.optimized.wasm}.wasm"' {} \; | ||
find . -type f ! -name "*.wasm" -delete | ||
|
||
- name: Upload artifact | ||
uses: actions/upload-artifact@v4 | ||
ahramy marked this conversation as resolved.
Show resolved
Hide resolved
|
||
with: | ||
name: ${{ steps.set-artifact-name.outputs.name }} | ||
path: target/wasm32-unknown-unknown/release | ||
retention-days: 1 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,37 +1,43 @@ | ||
name: "Publish specific rust package" | ||
name: "Upload Contract wasm to Cloudflare R2" | ||
|
||
on: | ||
workflow_call: | ||
inputs: | ||
# Package related variables | ||
package-name: | ||
description: "The package name to use (ex: gz-srv)" | ||
type: string | ||
required: true | ||
default: "" | ||
|
||
package-version: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why remove newlines? it's more readable that way |
||
description: "The package version to use (ex: 0.1.2)" | ||
type: string | ||
required: true | ||
default: "" | ||
|
||
package-git-tag: | ||
description: "the release tag name (ex: gz-srv-v0.1.2)" | ||
type: string | ||
required: true | ||
default: "" | ||
|
||
# CF Bucket related variables | ||
artifact-name: | ||
description: "Name of the artifact containing the builds" | ||
type: string | ||
required: true | ||
artifact-path: | ||
description: "Path of the artifact containing the builds" | ||
type: string | ||
required: true | ||
cf-bucket-name: | ||
description: "The CF bucket name to use" | ||
required: true | ||
type: string | ||
|
||
cf-config-bucket-root-key: | ||
description: "The root key to be used for accessing the configs. (ex: `test-root-key` puts releases in `test-root-key/*`)" | ||
required: true | ||
type: string | ||
github-release: | ||
description: "Whether to upload as a github release" | ||
type: boolean | ||
default: true | ||
|
||
secrets: | ||
github-token: | ||
|
@@ -48,36 +54,22 @@ on: | |
required: true | ||
|
||
jobs: | ||
build-and-upload: | ||
runs-on: blacksmith-8vcpu-ubuntu-2204 | ||
upload: | ||
name: Upload ${{ inputs.package-git-tag }} | ||
runs-on: blacksmith-2vcpu-ubuntu-2204 | ||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v4 | ||
with: | ||
fetch-depth: 0 | ||
token: ${{ secrets.github-token }} | ||
|
||
- name: Install Rust toolchain | ||
uses: dtolnay/rust-toolchain@stable | ||
- name: Download artifact | ||
uses: actions/download-artifact@v4 | ||
with: | ||
toolchain: 1.81.0 | ||
targets: wasm32-unknown-unknown | ||
name: ${{ inputs.artifact-name }} | ||
path: ${{ inputs.artifact-path }} | ||
|
||
- name: Build artifacts for ${{ inputs.package-name }}-v${{ inputs.package-version }} | ||
run: | | ||
echo "Building wasm for '${{ inputs.package-name }}-v${{ inputs.package-version }}'"; | ||
cargo install --locked stellar-cli --version 22.2.0 --features opt | ||
cargo wasm -p ${{ inputs.package-name }} | ||
stellar contract build | ||
./optimize.sh | ||
|
||
# Prepare the variables that will be used across the different next steps | ||
- name: Prepare cross-steps variables | ||
run: | | ||
export PACKAGE_NAME='${{ inputs.package-name }}' | ||
export PACKAGE_VERSION='v${{ inputs.package-version }}' | ||
export PACKAGE_VERSION=${{ inputs.github-release && format('v{0}', inputs.package-version) || inputs.package-version }} | ||
|
||
export BASE_ARTIFACTS_DIR="./target/wasm32-unknown-unknown/release" | ||
export BASE_ARTIFACTS_DIR='${{ inputs.artifact-path }}' | ||
export ARTIFACT_NAME="axelar-cgp-stellar-wasm-${PACKAGE_NAME}-${PACKAGE_VERSION}" | ||
export BASE_ARTIFACTS_VERSIONED_DIR="$(dirname ${BASE_ARTIFACTS_DIR})/${ARTIFACT_NAME}" # Regardless of the dir type, relative or absolute | ||
|
||
|
@@ -115,12 +107,6 @@ jobs: | |
# This cd to keep the dir structure of the artifacts archive | ||
cd ${{ env.BASE_ARTIFACTS_VERSIONED_DIR }} | ||
|
||
# Remove "unoptimized" built wasm files | ||
find "." -type f -name "*.wasm" ! -name "*.optimized.wasm" -maxdepth 1 -delete | ||
|
||
# Rename the optimized ones and remove the ".optimized" suffix | ||
find . -name "*.optimized.wasm" -maxdepth 1 -exec sh -c 'mv "$0" "${0%.optimized.wasm}.wasm"' {} \; | ||
|
||
# Archive the wasm | ||
find "." -type f -name "*.wasm" -maxdepth 1 -print | zip "${{ env.ZIP_ARCHIVE_FILE }}" -@ | ||
find "." -type f -name "*.wasm" -maxdepth 1 -print | tar -czvf "${{ env.TAR_ARCHIVE_FILE }}" -T - | ||
|
@@ -176,6 +162,7 @@ jobs: | |
# https://github.com/orgs/community/discussions/26263#discussioncomment-3251069 | ||
- name: Update the GitHub Release | ||
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191 # v2.0.8 | ||
if: inputs.github-release | ||
with: | ||
tag_name: ${{ inputs.package-git-tag }} # This uses the tag from the push | ||
files: | | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`# Build Stellar contracts on every commit to main and upload the wasm code to Cloudflare R2