Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Actions CI Port #16082

Open
wants to merge 16 commits into
base: master
Choose a base branch
from
+188 −7
Open

GitHub Actions CI Port #16082

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
720113e
GitHub Actions CI Port
illusion0001 Sep 15, 2024
91ef7e0
Add arm64 build, update Qt version and docker image for github actions
q4a Jan 19, 2025
9146b3f
Update rpcs3.yml
illusion0001 Jan 26, 2025
ff3ab91
Update rpcs3.yml
illusion0001 Jan 26, 2025
a914120
Update rpcs3.yml
illusion0001 Jan 26, 2025
3000bd5
Update rpcs3.yml
illusion0001 Jan 26, 2025
34f0b8e
Update rpcs3.yml
illusion0001 Jan 26, 2025
cb2464e
Update rpcs3.yml
illusion0001 Jan 26, 2025
e434a5e
Update rpcs3.yml
illusion0001 Jan 26, 2025
9cd3b58
Update rpcs3.yml
illusion0001 Jan 26, 2025
992b010
Update rpcs3.yml
illusion0001 Jan 26, 2025
dddd71f
Update rpcs3.yml
illusion0001 Jan 26, 2025
8eb3db7
Update rpcs3.yml
illusion0001 Jan 26, 2025
f734286
Update rpcs3.yml
illusion0001 Feb 2, 2025
e0a8cea
Update rpcs3.yml
illusion0001 Feb 2, 2025
5001988
Apply suggestions from code review
illusion0001 Feb 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .ci/build-linux-aarch64.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,16 @@ git submodule -q update --init $(awk '/path/ && !/llvm/ && !/opencv/ { print $3

mkdir build && cd build || exit 1

export CC="${CLANG_BINARY}"
export CXX="${CLANGXX_BINARY}"
if [ "$COMPILER" = "gcc" ]; then
# These are set in the dockerfile
export CC="${GCC_BINARY}"
export CXX="${GXX_BINARY}"
export LINKER=gold
export CFLAGS="-fuse-linker-plugin -fuse-ld=${LINKER}"
else
export CC="${CLANG_BINARY}"
export CXX="${CLANGXX_BINARY}"
fi

cmake .. \
-DCMAKE_INSTALL_PREFIX=/usr \
Expand Down
18 changes: 13 additions & 5 deletions .ci/setup-windows.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,16 +103,24 @@ COMM_HASH=$(git rev-parse --short=8 HEAD)
# Format the above into filenames
if [ -n "$PR_NUMBER" ]; then
AVVER="${COMM_TAG}-${COMM_HASH}"
BUILD="rpcs3-v${AVVER}_win64.7z"
BUILD_RAW="rpcs3-v${AVVER}_win64"
BUILD="${BUILD_RAW}.7z"
else
AVVER="${COMM_TAG}-${COMM_COUNT}"
BUILD="rpcs3-v${AVVER}-${COMM_HASH}_win64.7z"
BUILD_RAW="rpcs3-v${AVVER}-${COMM_HASH}_win64"
BUILD="${BUILD_RAW}.7z"
fi

# BRANCH is used for experimental build warnings for pr builds, used in main_window.cpp.
# BUILD is the name of the release artifact
# BUILD_RAW is just filename
# AVVER is used for GitHub releases, it is the version number.
BRANCH="${REPO_NAME}/${REPO_BRANCH}"
echo "BRANCH=$BRANCH" > .ci/ci-vars.env
echo "BUILD=$BUILD" >> .ci/ci-vars.env
echo "AVVER=$AVVER" >> .ci/ci-vars.env

# SC2129
{
echo "BRANCH=$BRANCH"
echo "BUILD=$BUILD"
echo "BUILD_RAW=$BUILD_RAW"
echo "AVVER=$AVVER"
} >> .ci/ci-vars.env
171 changes: 171 additions & 0 deletions .github/workflows/rpcs3.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
name: Build RPCS3

on:
push:
paths-ignore:
- '.cirrus.yml'
- '.azure-pipelines.yml'
- 'README.md'
pull_request:
paths-ignore:
- '.cirrus.yml'
- '.azure-pipelines.yml'
- 'README.md'
workflow_dispatch:

concurrency:
group: ${{ github.ref }}-${{ github.event_name }}
cancel-in-progress: true

env:
BUILD_REPOSITORY_NAME: ${{ github.repository }}
BUILD_SOURCEBRANCHNAME: ${{ github.ref_name }}
BUILD_ARTIFACTSTAGINGDIRECTORY: ${{ github.workspace }}/build

jobs:
Linux_Build:
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-24.04
docker_img: "rpcs3/rpcs3-ci-focal:1.9"
illusion0001 marked this conversation as resolved.
Show resolved Hide resolved
build_sh: "/rpcs3/.ci/build-linux.sh"
compiler: clang
- os: ubuntu-24.04
docker_img: "rpcs3/rpcs3-ci-focal:1.9"
illusion0001 marked this conversation as resolved.
Show resolved Hide resolved
build_sh: "/rpcs3/.ci/build-linux.sh"
compiler: gcc
- os: ubuntu-24.04-arm
docker_img: "rpcs3/rpcs3-ci-focal-aarch64:1.0"
illusion0001 marked this conversation as resolved.
Show resolved Hide resolved
build_sh: "/rpcs3/.ci/build-linux-aarch64.sh"
compiler: clang
runs-on: ${{ matrix.os }}
env:
CCACHE_DIR: ${{ github.workspace }}/ccache
CI_HAS_ARTIFACTS: true
DEPLOY_APPIMAGE: true
APPDIR: "/rpcs3/build/appdir"
ARTDIR: "/root/artifacts"
RELEASE_MESSAGE: "/rpcs3/GitHubReleaseMessage.txt"
COMPILER: ${{ matrix.compiler }}

steps:
- name: Checkout repository
uses: actions/checkout@main
with:
submodules: recursive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't want to checkout any submodules here since we do filtered checkout of submodules in the build script.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

fetch-depth: 0

- name: Setup Cache
uses: actions/cache@main
with:
path: ${{ env.CCACHE_DIR }}
key: ${{ runner.os }}-ccache-${{ matrix.compiler }}

- name: Docker setup and build
run: |
docker pull --quiet ${{ matrix.docker_img }}
docker run \
-v $PWD:/rpcs3 \
--env-file .ci/docker.env \
-v ${{ env.CCACHE_DIR }}:/root/.ccache \
-v ${{ github.workspace }}/artifacts:/root/artifacts \
${{ matrix.docker_img }} \
${{ matrix.build_sh }}

- name: Upload artifacts
#TODO: Upload artifact to release repository
#condition for release
#if: |
# github.event_name != 'pull_request' &&
# github.repository == 'RPCS3/rpcs3' &&
# github.ref == 'refs/heads/master' &&
# matrix.compiler == 'clang'
uses: actions/upload-artifact@main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with actions/upload-artifact@main, is this going to upload binaries to RPCS3/rpcs3-binaries-win/linux/linux-arm64/etc ? I don't see where we set any variables related to the upload repository, so I don't think this will work how we want it to.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also only run this step with the same restrictions we use in Azure:
condition: and(ne(variables['Build.Reason'], 'PullRequest'), eq(variables['Build.Repository.Name'], 'RPCS3/rpcs3'), eq(variables['Build.SourceBranch'], 'refs/heads/master'), eq(variables['COMPILER'], 'clang'))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added

        if: |
          github.event_name != 'pull_request' &&
          github.repository == 'RPCS3/rpcs3' &&
          github.ref == 'refs/heads/master' &&
          matrix.compiler == 'clang'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with actions/upload-artifact@main, is this going to upload binaries to RPCS3/rpcs3-binaries-win/linux/linux-arm64/etc ? I don't see where we set any variables related to the upload repository, so I don't think this will work how we want it to.

Nope, there will have to be a separate step for it. Using appropriate tokens like azure to upload it

with:
name: RPCS3 for Linux (${{ runner.arch }}, ${{ matrix.compiler }})
path: ${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/*.AppImage
compression-level: 0

Windows_Build:
runs-on: windows-latest
env:
COMPILER: msvc
QT_VER_MAIN: '6'
QT_VER: '6.8.1'
QT_VER_MSVC: 'msvc2022'
QT_DATE: '202411221531'
VULKAN_VER: '1.3.268.0'
VULKAN_SDK_SHA: '8459ef49bd06b697115ddd3d97c9aec729e849cd775f5be70897718a9b3b9db5'
CACHE_DIR: ./cache
steps:

- name: Checkout repository
uses: actions/checkout@main
with:
submodules: recursive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, we do a filtered submodule checkout in the setup script.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

fetch-depth: 0

- name: Setup env
run: |
echo "QTDIR=C:\Qt\${{ env.QT_VER }}\${{ env.QT_VER_MSVC }}_64" >> ${{ github.env }}
echo "VULKAN_SDK=C:\VulkanSDK\${{ env.VULKAN_VER }}" >> ${{ github.env }}

- name: Get Cache Keys
shell: bash
run: .ci/get_keys-windows.sh

- name: Setup Cache
uses: actions/cache@main
with:
path: ${{ env.CACHE_DIR }}
key: ${{ runner.os }}-${{ env.COMPILER }}-${{ env.QT_VER }}-${{ env.VULKAN_SDK_SHA }}-llvm.lock-glslang.lock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if the llvm.lock and glslang.lock keys are working? In Azure providing the filename means it will hash the file for the key. However, in https://github.com/actions/cache#creating-a-cache-key I see that they use this function: hashFiles('**/lockfiles') to hash files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will check

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me know what the outcome is here.

restore-keys: ${{ runner.os }}-${{ env.COMPILER }}

- name: Download and unpack dependencies
shell: bash
run: .ci/setup-windows.sh

- name: Export Variables
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should should be in a script file. Also, is it possible to do this step in bash?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be done in bash, as for making it a file, I'm afraid that github.env won't be resolved there, hence why it's a step.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh that's unfortunate, if you could rewrite in bash that'd be great for consistency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated this step to bash

run: |
Get-Content ".ci/ci-vars.env" | ForEach-Object {
# Skip over lines containing comments.
if ($_ -notmatch '^\s*#')
{
$line = $_ -split '='
$key = $line[0]
$val = $line[1]
echo "$key=$val" >> $env:GITHUB_ENV
}
}

- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@main

- name: Compile RPCS3
run: msbuild rpcs3.sln /p:Configuration=Release /p:Platform=x64

- name: Pack up build artifacts
shell: bash
run: |
mkdir -p ${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}
.ci/deploy-windows.sh

- name: Upload artifacts (7z)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same problems with the linux version of this step

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be sorted

#TODO: Upload artifact to release repository
#condition for release
#if: |
# github.event_name != 'pull_request' &&
# github.repository == 'RPCS3/rpcs3' &&
# github.ref == 'refs/heads/master'
uses: actions/upload-artifact@main
with:
name: RPCS3 for Windows (MSVC)
# 7z
# 7z.sha256
path: |
${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/${{ env.BUILD }}
${{ env.BUILD_ARTIFACTSTAGINGDIRECTORY }}/${{ env.BUILD }}.sha256
compression-level: 0
illusion0001 marked this conversation as resolved.
Show resolved Hide resolved
if-no-files-found: error