Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move code coverage from Travis to Scrutinizer #6556

Closed
mnapoli opened this issue Oct 30, 2014 · 4 comments
Closed

Move code coverage from Travis to Scrutinizer #6556

mnapoli opened this issue Oct 30, 2014 · 4 comments
Assignees
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone

Comments

@mnapoli
Copy link
Contributor

mnapoli commented Oct 30, 2014

We have enabled Scrutinizer for Piwik. This CI tools mainly does code quality inspections, which includes the classic ones (pdepend, copy paste detector, …) but also a proprietary one they have developed which is pretty powerful and useful.

Scrutinizer also offers to run tests and code coverage reports. For now I have set up code coverage for unit tests only: https://scrutinizer-ci.com/g/piwik/piwik

The goal is to remove code coverage completely from Travis so that builds are even faster.

I think we should not have code coverage for integration tests. If we agree on that, then we just have to remove code coverage in the Travis build.

If we decide to keep code coverage for integration tests, then we need to configure them on Scrutinizer, then we can remove them from Travis.

So should we have code coverage for integration tests?

@mnapoli mnapoli added Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. labels Oct 30, 2014
@mnapoli mnapoli self-assigned this Oct 30, 2014
@mattab
Copy link
Member

mattab commented Oct 31, 2014

+1 to remove coveralls (cc @sgiehl) as using scrutinizer will make our builds faster and provide interesting code quality reports

So should we have code coverage for integration tests?

I don't have strong opinion so if others agree we could do coverage for unit tests only.

@mattab mattab added this to the Piwik 2.10.0 milestone Oct 31, 2014
@sgiehl
Copy link
Member

sgiehl commented Oct 31, 2014

Do we already have a paid plan for scrutinizer? Otherwise it only supports sending external code coverage like coveralls does.
I'm not sure if it's good to remove coveralls.io. Currently we have the option to build code coverage for all plugins using coveralls.io. That can also be used (very easily) by third party plugins. Moving to scrutinizer will kill that possibility I guess, as we would remove the coveralls code.

@mnapoli
Copy link
Contributor Author

mnapoli commented Nov 2, 2014

@sgiehl for Scrutinizer we can run the tests (+code coverage) for free (as long as the repo is open source).

We just discussed some difficulties about plugins friday, what we suggest:

  • move piwik/piwik code coverage builds to Scrutinizer (so that Travis builds faster)
  • leave code coverage builds on Travis for plugins, but use the support for sending eternal code coverage to Scrutinizer
  • drop Coveralls, since code coverage can be handled on Scrutinizer (and Scrutinizer does more than Coveralls)

Moving to scrutinizer will kill that possibility I guess, as we would remove the coveralls code.

3rd party plugins can also use Scrutinizer since it's free?

@mattab
Copy link
Member

mattab commented Nov 3, 2014

@sgiehl Scrutiniser also provides other reports beyond coverage that are quite interesting, likely they would help the Piwik developer community (including us) increase the code quality of plugins 👍

followed up in #6931 Generate Unit tests code coverage report for Plugins

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Development

No branches or pull requests

3 participants