Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When generating a travis.yml file, add all jobs by default and do not fail if there are no tests #9274

Closed
tsteur opened this issue Nov 25, 2015 · 5 comments
Closed

When generating a travis.yml file, add all jobs by default and do not fail if there are no tests #9274

tsteur opened this issue Nov 25, 2015 · 5 comments
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone
Priority Backlog ...

Comments

@tsteur
Copy link
Member

tsteur commented Nov 25, 2015

See discussion in matomo-org/travis-scripts#19

When generating a travis-yml file via ./console generate-travis-yml there is by default no job for a UI test generated (unless the plugin already contains some UI tests).

When generating a UI test someone can execute them locally but they won't be executed on travis and it is easy to forget to add this check manually and to never notice the tests fail (happened to me with CustomDimensions plugin).

When generating a new UI test we should check whether a travis.yml exists, and if so add a line TEST_SUITE=UITests MYSQL_ADAPTER=PDO_MYSQL TEST_AGAINST_PIWIK_BRANCH=$PIWIK_TEST_TARGET to env/matrix in case the line is not present yet.
@tsteur tsteur added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. labels Nov 25, 2015
@tsteur
Copy link
Member Author

tsteur commented Nov 25, 2015

This is maybe even more important for Piwik PRO where a travis-yml file is generated automatically on plugin creation and when they add a new UI test afterwards it won't be executed on Travis I believe (unless one adds mentioned line to travis-yml manually)

@mattab
Copy link
Member

mattab commented Nov 25, 2015

+1 to always run the UI test CI job so that there is no issue in the future when one will add the UI test.

@mattab mattab added this to the 2.15.1 milestone Nov 25, 2015
@mattab mattab modified the milestone: 2.15.1 Dec 4, 2015
@mattab mattab added this to the 2.16.0 milestone Dec 23, 2015
@diosmosis
Copy link
Member

FYI, this is not an issue for pro plugins w/ autogenerated .travis.yml files. UI test builds are forced for those plugins. Can be an issue, if the build is removed manually.

diosmosis pushed a commit that referenced this issue Dec 26, 2015
Refs #9274, add logic to generate:test command to add travis relevant…
abe144f 
… travis job(s) when .travis.yml file is present, the test target env var can be deduced and the job is missing from the matrix.

Notes:
- Uses regexes to preserve original formatting.
- Additions appear to be well formed as long as initial .travis.yml is well formed.
@diosmosis diosmosis mentioned this issue Dec 26, 2015
Closed
@mattab mattab modified the milestones: 2.16.1, 2.16.0 Jan 27, 2016
@mattab mattab modified the milestones: 2.16.1, 3.0.0 Feb 3, 2016
@tsteur tsteur changed the title When generating a UI test, add a UI test job to travis.yml When generating a travis.yml file, add all jobs by default and do not fail if there are no tests Aug 30, 2016
@tsteur
Copy link
Member Author

tsteur commented Aug 30, 2016

Changing title from When generating a UI test, add a UI test job to travis.yml"" to "When generating a travis.yml file, add all jobs by default and do not fail if there are no tests".

I think it's better to always run all jobs this would make it easier so we do not have to add any jobs later etc and might be even more encouraging to write ui tests etc. Maybe we could even create a default UI test with every plugin.

In tests/travis/generator/Generator/PluginTravisYmlGenerator.php it would be easily to always generate all jobs by eg not checking for isTargetPluginContainsUITests but I tested it and it would fail the build if there are eg no UI tests for a plugin.

@tsteur
Copy link
Member Author

tsteur commented Jan 3, 2024

Closing as we're now using Github actions

@tsteur tsteur closed this as not planned Won't fix, can't repro, duplicate, stale Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Milestone
Priority Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@diosmosis @tsteur @mattab