@tsteur opened this Issue on November 25th 2015 Member

See discussion in https://github.com/piwik/travis-scripts/issues/19

When generating a travis-yml file via ./console generate-travis-yml there is by default no job for a UI test generated (unless the plugin already contains some UI tests).

When generating a UI test someone can execute them locally but they won't be executed on travis and it is easy to forget to add this check manually and to never notice the tests fail (happened to me with CustomDimensions plugin).

When generating a new UI test we should check whether a travis.yml exists, and if so add a line TEST_SUITE=UITests MYSQL_ADAPTER=PDO_MYSQL TEST_AGAINST_PIWIK_BRANCH=$PIWIK_TEST_TARGET to env/matrix in case the line is not present yet.

@tsteur commented on November 25th 2015 Member

This is maybe even more important for Piwik PRO where a travis-yml file is generated automatically on plugin creation and when they add a new UI test afterwards it won't be executed on Travis I believe (unless one adds mentioned line to travis-yml manually)

@mattab commented on November 25th 2015 Member

+1 to always run the UI test CI job so that there is no issue in the future when one will add the UI test.

@diosmosis commented on December 26th 2015 Member

FYI, this is not an issue for pro plugins w/ autogenerated .travis.yml files. UI test builds are forced for those plugins. Can be an issue, if the build is removed manually.

@tsteur commented on August 30th 2016 Member

Changing title from When generating a UI test, add a UI test job to travis.yml"" to "When generating a travis.yml file, add all jobs by default and do not fail if there are no tests".

I think it's better to always run all jobs this would make it easier so we do not have to add any jobs later etc and might be even more encouraging to write ui tests etc. Maybe we could even create a default UI test with every plugin.

In tests/travis/generator/Generator/PluginTravisYmlGenerator.php it would be easily to always generate all jobs by eg not checking for isTargetPluginContainsUITests but I tested it and it would fail the build if there are eg no UI tests for a plugin.

Powered by GitHub Issue Mirror