Showing coverage info on GitHub repos?

[sanjayankur31]

Hi folks,

What setup/config/services are you folks using to get/show converage information on your GitHub repos? I wanted to enable it on our NeuroML repos, especially the Python bits where we use pytest for testing.

I see lots of posts on doing this, but if you have a working setup already, I'll just adapt (cough steal cough) that :)

[sanjayankur31]

tagging the WG, not sure everyone gets notifications otherwise: @OCNS/software-wg

[mstimberg]

For https://github.com/brian-team/brian2, we use https://coveralls.io (see https://github.com/brian-team/brian2/blob/master/.github/workflows/testsuite.yml) – I think instead of installing/using the separate coverage tool, nowadays, one would instead rather use the pytest-cov plugin. We only use it for displaying a badge in the README, though, i.e. we do not have it post comments in pull requests, etc.

[neworderofjamie]

GeNN uses codecov.io https://github.com/genn-team/genn to generate combined C++ and Python coverage (e.g. https://app.codecov.io/github/genn-team/genn?branch=master). We use pytest-cov for the Python bit

[sanjayankur31]

Cool. We have pytest-cov too, but I don't think we've configured it correctly to run on our CI. BTW, are both coveralls and codecov free for public/open projects (or did you have to pay to use them)?

[mstimberg]

coveralls.io is free for open-source: https://coveralls.io/pricing

[neworderofjamie]

As is codecov.io - no idea of the pros and cons between them! Our CI is a bit of a Jenkins mess at the moment but it uploads using the bash up loader like https://github.com/genn-team/genn/blob/80dd174f5133ce0c9287cfdc1b2de46eb340b4e0/Jenkinsfile#L248

[sanjayankur31]

Thanks. I had a quick look at some comparison posts etc. Not a lot between them from the look of it. Also ran into https://www.codacy.com/pricing

I'll go set one up, thanks both :)

[sanjayankur31]

I set it up now. Here's a PR:

NeuroML/pyNeuroML#387

It doesn't use coveralls/codecov/codacy. There are two github actions that:

• parse the coverage from pytest-cov and spit it out in a markdown file
• posts this as a single comment to the PR

Seems to work well enough (and shows that some of our legacy code could use more testing to improve coverage..)

[sanjayankur31]

This is the action: https://github.com/marketplace/actions/code-coverage-summary

also has an example with the commenting bit