Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CodeBuild Runner integration to include the run id and attempt #738

Closed
wants to merge 33 commits into from
Closed
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
13a5164
Update CodeBuild Runner integration to include the run id and attempt
popematt Apr 9, 2024
4288699
Attempt to clean up the strategy matrix
popematt Apr 9, 2024
70cd833
Fix matrix os to be json
popematt Apr 9, 2024
9f9cbe2
Another attempt to fix matrix os
popematt Apr 9, 2024
de69949
Fix var context reference
popematt Apr 9, 2024
9967635
Fix format function syntax
popematt Apr 9, 2024
ddf28ba
Refactor runner selection logic into separate step
popematt Apr 9, 2024
f01244f
Pass job outputs as serialized json
popematt Apr 9, 2024
df7e278
Try to fix deserialization of runner types
popematt Apr 9, 2024
48ab2a7
Fix logic error
popematt Apr 9, 2024
bea469a
Try to fix use-codebuild flag
popematt Apr 9, 2024
95c5d80
Add step to dump runner config
popematt Apr 9, 2024
002596e
Finished the runner setup
popematt Apr 10, 2024
f0abe34
Fix build confirmation step
popematt Apr 10, 2024
d7ce4d4
Fix al2 runner label outputs
popematt Apr 10, 2024
c18587b
Fix typo in build matrix
popematt Apr 10, 2024
5bf4b74
Re-add 'if: always()' to build confirmation job
popematt Apr 10, 2024
c9c3c5a
Fix codebuild project name check
popematt Apr 10, 2024
dc63443
Fix setup step
popematt Apr 10, 2024
8000b76
Try referencing property of deserialized json object
popematt Apr 10, 2024
59c4b29
Add missing commas, fix job result path
popematt Apr 10, 2024
6aa208f
Try referencing property of deserialized json object again
popematt Apr 10, 2024
bc4876e
Try referencing property of deserialized json object (2)
popematt Apr 10, 2024
68b06ea
Try referencing property of deserialized json object (3)
popematt Apr 10, 2024
1c7d04a
Try referencing property of deserialized json object (4)
popematt Apr 10, 2024
ba19f46
Try referencing property of deserialized json object (5)
popematt Apr 10, 2024
87f9363
Try referencing property of deserialized json object (6)
popematt Apr 10, 2024
87545b7
Try referencing property of deserialized json object (7)
popematt Apr 10, 2024
f650cd2
Try referencing property of deserialized json object (8)
popematt Apr 10, 2024
7157f66
Try referencing property of deserialized json object (9)
popematt Apr 10, 2024
24251df
Try referencing property of deserialized json object (10)
popematt Apr 10, 2024
af35bba
Try referencing property of deserialized json object (11)
popematt Apr 10, 2024
04ac970
Try referencing property of deserialized json object (12)
popematt Apr 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 62 additions & 17 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
@@ -1,27 +1,61 @@
name: CI Build

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Setup Build Matrix (true, ion-rust, windows, macos, ubuntu, al2-intel, al2-arm, windows, macos, u...

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (windows, default)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (windows, all)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (windows, experimental-ion-hash)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (windows, experimental)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (macos, default)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (macos, all)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (macos, experimental-ion-hash)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (macos, experimental)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, default)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, all)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, experimental-ion-hash)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (ubuntu, experimental)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-intel, default)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-intel, all)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-intel, experimental-ion-hash)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-intel, experimental)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-arm, default)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-arm, all)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-arm, experimental-ion-hash)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test (al2-arm, experimental)

.github/workflows/rust.yml#L1

This run was manually canceled.

Check failure on line 1 in .github/workflows/rust.yml

View workflow job for this annotation

GitHub Actions / Build and Test Confirmation

.github/workflows/rust.yml#L1

This run was manually canceled.

on: [push, pull_request]

jobs:
setup:
# This job sets up the runners to be used in the matrix for the build workflow.
# It provides a list of available runners with stable, human-friendly names and a mapping
# from those names to the actual `runs-on` value for each runner type. This allows us to
# use codebuild-hosted runners for amazon-ion/ion-rust without requiring forks to also
# have codebuild-hosted runners.
#
# If you want to use codebuild runners for your personal fork, follow the instructions to set
# up a codebuild project. https://docs.aws.amazon.com/codebuild/latest/userguide/action-runner.html
# Then, create a repository variable for your fork named `CODEBUILD_PROJECT_NAME` with the name
# of the project you created.
#
# TODO: See if this job can be turned into a reusable component
name: Setup Build Matrix
runs-on: ubuntu-latest
strategy:
matrix:
# We're using a matrix with a single entry so that we can define some config as YAML rather than
# having to write escaped json in a string.
include:
# Repository vars are not available for fork->source pull requests, so if we want to use codebuild runners
# for PRs, we need a fallback to check the repository owner.
- use-codebuild: ${{ vars.CODEBUILD_PROJECT_NAME != '' || github.repository_owner == 'amazon-ion' }}
codebuild-project-name: ${{ vars.CODEBUILD_PROJECT_NAME != '' && vars.CODEBUILD_PROJECT_NAME || 'ion-rust' }}
runs-on-names-cb: [ windows, macos, ubuntu, al2-intel, al2-arm ]
runs-on-names: [ windows, macos, ubuntu ]
runner-labels:
windows: windows-latest
ubuntu: ubuntu-latest
macos: macos-latest
al2-intel: "codebuild-${{ vars.CODEBUILD_PROJECT_NAME != '' && vars.CODEBUILD_PROJECT_NAME || 'ion-rust' }}-${{ github.run_id }}-${{ github.run_attempt }}-al2-5.0-large"
al2-arm: "codebuild-${{ vars.CODEBUILD_PROJECT_NAME != '' && vars.CODEBUILD_PROJECT_NAME || 'ion-rust' }}-${{ github.run_id }}-${{ github.run_attempt }}-arm-3.0-large"
outputs:
available-runners: ${{ matrix.use-codebuild && toJSON(matrix.runs-on-names-cb) || toJSON(matrix.runs-on-names) }}
runner-labels: ${{ toJSON(matrix.runner-labels) }}
steps:
- name: Dump Config
run: echo '${{ toJSON(matrix) }}'

build:
name: Build and Test
runs-on: ${{ matrix.os }}
# We want to run on external PRs, but not on internal ones as push automatically builds
# H/T: https://github.com/Dart-Code/Dart-Code/commit/612732d5879730608baa9622bf7f5e5b7b51ae65
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'amazon-ion/ion-rust'
needs: setup
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest, codebuild-ion-rust-arm-3.0-large, codebuild-ion-rust-al2-5.0-large ]
# use the available runner types that were determined by the setup step
os: ${{ fromJSON(needs.setup.outputs.available-runners) }}
# build and test for different and interesting crate features
features: ['default', 'all', 'experimental-ion-hash', 'experimental']
# Forks cannot use the codebuild runners. The job will hang indefinitely waiting to be picked up by a worker.
isFork:
- ${{ github.repository_owner != 'amazon-ion' }}
exclude:
- isFork: true
os: codebuild-ion-rust-arm-3.0-large
- isFork: true
os: codebuild-ion-rust-al2-5.0-large
features: [ 'default', 'all', 'experimental-ion-hash', 'experimental' ]
# Map the friendly names from `available-runners` to the actual runner labels.
runs-on: ${{ fromJSON(needs.setup.outputs.runner-labels)[matrix.os] }}
# We want to run on external PRs, but not on internal ones as push automatically builds
# H/T: https://github.com/Dart-Code/Dart-Code/commit/612732d5879730608baa9622bf7f5e5b7b51ae65
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != 'amazon-ion/ion-rust'
permissions:
checks: write

Expand Down Expand Up @@ -74,10 +108,10 @@
# The clippy check depends on setup steps defined above, but we don't want it to run
# for every OS because it posts its comments to the PR. These `if` checks limit clippy to
# only running on the Linux test. (The choice of OS was arbitrary.)
if: matrix.os == 'ubuntu-latest' && matrix.features == 'all'
if: matrix.os == 'ubuntu' && matrix.features == 'all'
run: rustup component add clippy
- name: Run Clippy
if: matrix.os == 'ubuntu-latest' && matrix.features == 'all'
if: matrix.os == 'ubuntu' && matrix.features == 'all'
uses: actions-rs/clippy-check@v1
with:
# Adding comments to the PR requires the GITHUB_TOKEN secret.
Expand All @@ -87,8 +121,19 @@
args: --workspace --all-features --tests -- -Dwarnings
- name: Rustdoc on Everything
# We really only need to run this once--ubuntu/all features mode is as good as any
if: matrix.os == 'ubuntu-latest' && matrix.features == 'all'
if: matrix.os == 'ubuntu' && matrix.features == 'all'
uses: actions-rs/cargo@v1
with:
command: doc
args: --document-private-items --all-features
confirm-build:
# This job is just a "join" on all parallel strategies for the `build` job so that we can require it in our branch protection rules.
needs: build
name: Build and Test Confirmation
runs-on: ubuntu-latest
if: always()
steps:
- run: echo ${{ needs.build.result }}
- if: needs.build.result != 'success'
run: exit 1

Loading