Skip to content

Commit

Permalink
Move benchmark workflow out of main tox action into codspeed
Browse files Browse the repository at this point in the history
Let's see how this goes
  • Loading branch information
SolarDrew committed May 8, 2024
1 parent fae780a commit 1807001
Show file tree
Hide file tree
Showing 3 changed files with 19 additions and 15 deletions.
19 changes: 19 additions & 0 deletions .github/workflows/codspeed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: codspeed-benchmarks

on:
push:
branches:
- "main"
pull_request:
workflow_dispatch:

jobs:
benchmarks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run benchmarks
uses: CodspeedHQ/action@v2
with:
token: ${{ secrets.CODSPEED_TOKEN }}
run: "pytest -vvv -r fEs --pyargs dkist -k test_benchmarks --benchmark-autosave"
1 change: 0 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ jobs:
- windows: py311-online
- macos: py310
- linux: py310-oldestdeps
- linux: benchmarks
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

Expand Down
14 changes: 0 additions & 14 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -90,17 +90,3 @@ commands =
!notebooks: sphinx-build -j 1 --color -W --keep-going -b html -d _build/.doctrees . _build/html -D nb_execution_mode=off {posargs}
notebooks: sphinx-build -j 1 --color -W --keep-going -b html -d _build/.doctrees . _build/html {posargs}
python -c 'import pathlib; print("Documentation available under file://\{0\}".format(pathlib.Path(r"{toxinidir}") / "docs" / "_build" / "index.html"))'

[testenv:benchmarks]
description = Run benchmarks on PR and compare against main to ensure there are no performance regressions
allowlist_externals=git
commands =
git clone https://github.com/DKISTDC/dkist-benchmarks.git {env_tmp_dir}/dkist-benchmarks
{env:PYTEST_COMMAND} -k test_benchmarks --benchmark-autosave --benchmark-compare={env_tmp_dir}/dkist-benchmarks/main.json --benchmark-compare-fail=mean:5%

[testenv:benchmark-main]
description = Run benchmarks on main to keep comparison data up to date
allowlist_externals = git,cp
commands =
{env:PYTEST_COMMAND} -k test_benchmarks --benchmark-save=main
cp {toxinidir}/.tmp/benchmark-main/.benchmarks/Linux-CPython-3.10-64bit/0001_main.json /github/main.json

0 comments on commit 1807001

Please sign in to comment.