This repository has been archived by the owner on Nov 25, 2022. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
replace travis with circle-ci, cleanup of build steps (#639)
Travis is getting slower and was already hard to debug. This commit: - moves from travis to Circle-CI - has a ci-scripts/README.md which explains what is contained there - puts all build and CI scripts into "ci_scripts" directory - allows to run full ci runs locally - trims down the base Docker image where tests/docs/wheels are run from 4.4GB to 1.7GB - fixes #628 Using circle-ci also means that we can probably integrate Mac-OS build steps more easily. I am sure that my initial circle-ci config (helped by kind hints from @dignifiedquire ) can be improved.
- Loading branch information
Showing
34 changed files
with
608 additions
and
309 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
../ci_scripts/config.yml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
|
||
# Continuous Integration Scripts for Delta Chat | ||
|
||
Continuous Integration is run through CircleCI | ||
but is largely independent of it. | ||
|
||
|
||
## Generating docker containers for performing build step work | ||
|
||
All tests, docs and wheel building is run in docker containers: | ||
|
||
- **coredeps/Dockerfile** specifies an image that contains all | ||
of Delta Chat's core dependencies as linkable libraries. | ||
It also serves to run python tests and build wheels | ||
(binary packages for Python). | ||
|
||
- **doxygen/Dockerfile** specifies an image that contains | ||
the doxygen tool which is used to generate C-docs. | ||
|
||
To run tests locally you can pull existing images from "docker.io", | ||
the hub for sharing Docker images:: | ||
|
||
docker pull deltachat/coredeps | ||
docker pull deltachat/doxygen | ||
|
||
or you can build the docker images yourself locally | ||
to avoid the relatively large download:: | ||
|
||
cd ci_scripts # where all CI things are | ||
docker build -t deltachat/coredeps docker-coredeps | ||
docker build -t deltachat/doxygen docker-doxygen | ||
|
||
## ci_run.sh (main entrypoint called by circle-ci) | ||
|
||
Once you have the docker images available | ||
you can run python testing, documentation generation | ||
and building binary wheels:: | ||
|
||
sh DOCS=1 TESTS=1 ci_scripts/ci_run.sh | ||
|
||
## ci_upload.sh (uploading artifacts on success) | ||
|
||
- python docs to `https://py.delta.chat/_unofficial_unreleased_docs/<BRANCH>` | ||
|
||
- doxygen docs to `https://c.delta.chat/_unofficial_unreleased_docs/<BRANCH>` | ||
|
||
- python wheels to `https://m.devpi.net/dc/<BRANCH>` | ||
so that you install fully self-contained wheels like this: | ||
`pip install -U -i https://m.devpi.net/dc/<BRANCH> deltachat` | ||
|
||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# perform CI jobs on PRs and after merges to master. | ||
# triggered from .circleci/config.yml | ||
|
||
set -e -x | ||
|
||
export BRANCH=${CIRCLE_BRANCH:-test7} | ||
|
||
# run doxygen on c-source (needed by later doc-generation steps). | ||
# XXX modifies the host filesystem docs/xml and docs/html directories | ||
# XXX which you can then only remove with sudo as they belong to root | ||
|
||
if [ -n "$DOCS" ] ; then | ||
docker run --rm -it -v $PWD:/mnt -w /mnt/docs deltachat/doxygen doxygen | ||
fi | ||
|
||
# run everything else inside docker (TESTS, DOCS, WHEELS) | ||
docker run -e BRANCH -e MESONARGS -e TESTS -e DOCS \ | ||
--rm -it -v $(pwd):/mnt -w /mnt \ | ||
deltachat/coredeps ci_scripts/run_all.sh | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
#!/bin/bash | ||
|
||
if [ -z "$DEVPI_LOGIN" ] ; then | ||
echo "required: password for 'dc' user on https://m.devpi/net/dc index" | ||
exit 1 | ||
fi | ||
|
||
set -xe | ||
|
||
DOXYDOCDIR=${1:?directory where doxygen docs to be found} | ||
PYDOCDIR=${2:?directory with python docs} | ||
WHEELHOUSEDIR=${3:?directory with pre-built wheels} | ||
|
||
export BRANCH=${CIRCLE_BRANCH:?specify branch for uploading purposes} | ||
|
||
|
||
# python docs to py.delta.chat | ||
rsync -avz \ | ||
-e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" \ | ||
"$PYDOCDIR/html/" \ | ||
[email protected]:build/${BRANCH} | ||
|
||
# C docs to c.delta.chat | ||
rsync -avz \ | ||
-e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" \ | ||
"$DOXYDOCDIR/html/" \ | ||
[email protected]:build-c/${BRANCH} | ||
|
||
echo ----------------------- | ||
echo upload wheels | ||
echo ----------------------- | ||
|
||
# Bundle external shared libraries into the wheels | ||
pushd $WHEELHOUSEDIR | ||
|
||
pip install devpi-client | ||
devpi use https://m.devpi.net | ||
devpi login dc --password $DEVPI_LOGIN | ||
|
||
devpi use dc/$BRANCH || { | ||
devpi index -c $BRANCH | ||
devpi use dc/$BRANCH | ||
} | ||
devpi index $BRANCH bases=/root/pypi | ||
devpi upload deltachat*.whl | ||
|
||
popd |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
|
||
version: 2 | ||
jobs: | ||
build_test_docs_wheel: | ||
machine: True | ||
steps: | ||
- checkout | ||
- run: docker pull deltachat/doxygen | ||
- run: docker pull deltachat/coredeps | ||
- run: | ||
command: ci_scripts/ci_run.sh | ||
environment: | ||
MESONARGS: | ||
DOCS: 1 | ||
TESTS: 1 | ||
|
||
- run: | ||
name: copying C docs, python docs and wheels to workspace | ||
command: | | ||
mkdir -p workspace/python | ||
cp -av docs workspace/c-docs | ||
cp -av python/.docker-tox/wheelhouse workspace/ | ||
cp -av python/doc/_build/ workspace/py-docs | ||
- persist_to_workspace: | ||
root: workspace | ||
paths: | ||
- c-docs | ||
- py-docs | ||
- wheelhouse | ||
|
||
upload_docs_wheels: | ||
machine: True | ||
steps: | ||
- run: | ||
command: | | ||
echo 'export MESONARGS=""' >> $BASH_ENV | ||
- checkout | ||
- attach_workspace: | ||
at: workspace | ||
- run: ls -laR workspace | ||
- run: ci_scripts/ci_upload.sh workspace/c-docs workspace/py-docs workspace/wheelhouse | ||
|
||
build_monolith_tests: | ||
machine: True | ||
steps: | ||
- checkout | ||
- run: docker pull deltachat/coredeps | ||
- run: | ||
command: | | ||
echo 'export MESONARGS="-Dmonolith=true"' >> $BASH_ENV | ||
echo 'export TESTS=1' >> $BASH_ENV | ||
- run: ci_scripts/ci_run.sh | ||
|
||
build_static: | ||
machine: True | ||
steps: | ||
- checkout | ||
- run: docker pull deltachat/coredeps | ||
- run: | ||
command: | | ||
echo 'export MESONARGS="--default-library=static"' >> $BASH_ENV | ||
- run: ci_scripts/ci_run.sh | ||
|
||
build_forcefallback: | ||
machine: True | ||
steps: | ||
- checkout | ||
- run: docker pull deltachat/coredeps | ||
- run: | ||
command: | | ||
echo 'export MESONARGS="--wrap-mode=forcefallback --default-library=static"' >> $BASH_ENV | ||
- run: ci_scripts/ci_run.sh | ||
|
||
workflows: | ||
version: 2 | ||
build_all: | ||
jobs: | ||
- build_test_docs_wheel | ||
- build_monolith_tests | ||
- build_static | ||
- build_forcefallback | ||
- upload_docs_wheels: | ||
requires: | ||
- build_test_docs_wheel | ||
- build_monolith_tests | ||
- build_static | ||
- build_forcefallback | ||
|
||
|
Oops, something went wrong.