Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Keeping CI builds green at all times for all our repositories #6544

Closed
3 tasks done
mattab opened this issue Oct 28, 2014 · 13 comments
Closed
3 tasks done

Keeping CI builds green at all times for all our repositories #6544

mattab opened this issue Oct 28, 2014 · 13 comments
Assignees
@diosmosis
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. RFC Indicates the issue is a request for comments where the author is looking for feedback. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone
2.12.0

Comments

@mattab
Copy link
Member

mattab commented Oct 28, 2014

The goal of this issue is that every week we want our 40+ projects CI builds to be green with as little pain and effort as possible. Having green builds across the board will increase quality of our work in general, increase developer happiness and help prevent us from deploying broken code.

Scope: this issue covers builds for our official Piwik Plugins (VisitorGenerator, CustomAlerts, etc.) and components and all Piwik PRO plugins.

Proposed steps For The Win

  1. Make all builds green on https://ci-status.com (at east once!)
  2. Automatically trigger the build for each repository once a week Automatically refresh every week the last CI build for each of our repository  #6542
  3. in CI job against latest stable, allow failures when plugin requires a recent beta Plugin CI should be "allow failures", when plugin requires a recent beta, in CI job TEST_AGAINST_CORE=latest_stable #6930

Related to

  1. Detect automatically when plugins use code that is @deprecated or not tagged with @api Check if plugin uses Piwik methods that are Deprecated or not API #6539

What we already have today

  • Plugins run tests against piwik/piwik master, to check pro-actively that plugins work with bleeding edge Piwik core platform
  • Plugins run tests against the latest stable release, to check that plugins work with current stable Piwik release.
  • Plugins can include their own Unit / Integration / Systems tests and UI Screenshot tests
  • Each developer is responsible for fixing the build after he broke the build
  • Piwik core developers are responsible to fix failures in a plugin when the failure is caused by a BC break or change in core platform (when it occurs then plugin developer should create an issue on piwik/piwik issue tracker to notify core developers)

What else can we do to help us achieve green builds every week on 40+ repositories?
@mattab mattab added Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. RFC Indicates the issue is a request for comments where the author is looking for feedback. labels Oct 28, 2014
@mattab mattab added this to the Piwik 2.10.0 milestone Oct 28, 2014
@mnapoli
Copy link
Contributor

mnapoli commented Oct 28, 2014

And we discussed "Detect automatically when plugins use code tagged with @deprecated"?

@mattab
Copy link
Member Author

mattab commented Oct 28, 2014

@mnapoli Good point, it will be included in scope of #6539

mnapoli added a commit to matomo-org/plugin-LoginLdap that referenced this issue Oct 29, 2014
@mnapoli
matomo-org/matomo#6544 Added badges
3657a82
mnapoli added a commit to matomo-org/plugin-PiwikDebugger that referenced this issue Oct 29, 2014
@mnapoli
matomo-org/matomo#6544 Added badge
92dd485
mnapoli added a commit to matomo-org/plugin-VisitorGenerator that referenced this issue Oct 29, 2014
@mnapoli
matomo-org/matomo#6544 Added badge
26f7a68
@mnapoli mnapoli mentioned this issue Oct 30, 2014
Closed
@mattab mattab modified the milestones: Piwik 2.10.0 , Piwik 2.11.0 Dec 3, 2014
@diosmosis diosmosis self-assigned this Dec 15, 2014
@mattab
Copy link
Member Author

mattab commented Dec 19, 2014

Current CI status: 26 builds failing out of 50 builds.

@mattab
Copy link
Member Author

mattab commented Jan 5, 2015

Current CI status: 17 builds failing out of 45 builds.

@mattab mattab mentioned this issue Jan 6, 2015
Closed
@mattab
Copy link
Member Author

mattab commented Jan 7, 2015

it's starting to look pretty green on CI status! 5 builds failing out of 45 builds

@mattab
Copy link
Member Author

mattab commented Jan 7, 2015

CI Status: 9 builds failing out of 50 builds.

@mattab
Copy link
Member Author

mattab commented Jan 11, 2015

after the automatic Sunday Builds refresh we have: 12 builds failing out of 50 builds.

@diosmosis diosmosis mentioned this issue Jan 12, 2015
Closed
4 tasks
@mattab
Copy link
Member Author

mattab commented Feb 5, 2015

CI Status: 9 builds failing out of 49 builds.

@mattab
Copy link
Member Author

mattab commented Feb 9, 2015

we're almost there! Ci Status: 6 builds failing out of 49 builds.

@mattab mattab modified the milestones: Piwik 2.11.0, Piwik 2.12.0 Feb 11, 2015
@mattab
Copy link
Member Author

mattab commented Feb 27, 2015

This is getting really exciting! Summary: 2 builds failing out of 47 builds.

This was referenced Mar 4, 2015
Closed
Closed
Closed
@mattab mattab mentioned this issue Mar 12, 2015
Open
@mattab
Copy link
Member Author

mattab commented Mar 13, 2015

Yesterday, 10 builds failing. Today 4 builds failing.

This new feature in ci-status will get rid of 2 failed builds: https://github.com/piwik/ci-status/issues/20

@mattab
Copy link
Member Author

mattab commented Mar 13, 2015

When following issues are closed we can close this issue:

The next step will be:

  • Decide who is responsible to fix the builds so that they are green each monday.

maybe someone from the team will volunteer to "own this" (eg. for a few months), or maybe a person rotating each week or month?

@mattab
Copy link
Member Author

mattab commented Mar 17, 2015

Created follow up issue: Decide who is responsible to fix the builds so that they are green each monday #7453

Well done to us, for the huge progress made in this area in the last few months... 👍

@mattab mattab closed this as completed Mar 17, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diosmosis diosmosis

Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. RFC Indicates the issue is a request for comments where the author is looking for feedback. Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Milestone
2.12.0
Development

No branches or pull requests
3 participants
@diosmosis @mattab @mnapoli