Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automated test that helps create awesome plugins #10278

Closed
16 tasks done
mattab opened this issue Jul 7, 2016 · 2 comments
Closed
16 tasks done

Automated test that helps create awesome plugins #10278

mattab opened this issue Jul 7, 2016 · 2 comments
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.

Comments

@mattab
Copy link
Member

mattab commented Jul 7, 2016

Plugin automated tests check

  • Check the plugin has integration + system tests defined in tests/

If the plugin defines some UI (ie. a directory templates/ or javascripts or angularjs is present)...

  • Check that there is at least one UI test defined
  • Check that plugin includes at least one screenshot image file in screenshots/ folder

Plugin communication

  • Check that Github repository has a description set
  • Check that plugin.json description does not contain TODO (the plugin description says TODO when it wasn't detected in 58ce312 )
  • Check that plugin.json description field is the same as the Github repository description

Plugin README file

  • Check that README has at least the following sections: Description, Changelog, Support
  • Check that README includes the travis build badge with status on master branch.
    • if plugin uses git flow, check that the README also includes the badge on develop branch

Check that README > Support section information is correctly set:

  • for open source plugins: link to the plugin issue tracker
  • for both: add a sentence to invite clients/users to report issues such as We welcome your feedback about this plugin. Please report bugs or suggest enhancements at %s

plugin.json checks

  • Make sure small errors are detected automatically: the plugin name in plugin.json must == the namespace name, otherwise autoloading errors appear after installing a plugin.
  • If there is a composer.json file, check that the version is the version in plugin.json
  • for plugins in Piwik github organisation, and homepage is http://plugins.piwik.org/PLUGIN_NAME, authors.name is Piwik, authors.email is hello@piwik.org and authors.homepage is http://piwik.org
  • for all open source plugins, check that license is GPL v3+

Plugin files

Check that PHP and JS files have the proper header set.

  • For open source code, the header is:
/**
 * Piwik - free/libre analytics platform
 *
 * @link http://piwik.org
 * @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
 */

Not included in MVP

these could be done later:

  • Plugins should also include architectural docs so future devs can understand how the plugin works and what it does w/o digging through confluence or the code base.
  • Automatically test reports using Report class metadata. Instead of manually designing system tests, we can probably test all reports w/ preset set of cases (eg, multiple sites, segment, multiple dates, range, different periods, etc.).
  • Check that plugin has composer.json file (or: Rename plugin.json to use composer.json  #5921 would be better)
  • check if the plugin has some logging statement and if not, output a warning or recommendation to add logging statements?

Pull request

A pull request was created implementing most of this, and is pending review/merge: https://github.com/piwik/travis-scripts/pull/18/files

@mattab mattab added the c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. label Jul 7, 2016
@mattab mattab added this to the Mid term milestone Jul 7, 2016
@hpvd
Copy link

hpvd commented Jul 8, 2016

maybe this could be also be covered/solved with these tests:
Name, descriptions etc of plugins uncomplete / got partly cut/hidden
#10255

?

@mattab
Copy link
Member Author

mattab commented Feb 20, 2017

not needed anymore

@mattab mattab closed this as completed Feb 20, 2017
@mattab mattab added the wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. label Feb 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

2 participants