New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Random UI test failures (due to bug in phantomjs2) #10536
Labels
answered
For when a question was asked and we referred to forum or answered it.
c: Tests & QA
For issues related to automated tests or making it easier to QA & test issues.
Milestone
Comments
mattab
added
the
c: Tests & QA
For issues related to automated tests or making it easier to QA & test issues.
label
Sep 22, 2016
mattab
added a commit
that referenced
this issue
Sep 22, 2016
mattab
added a commit
that referenced
this issue
Sep 22, 2016
This bug: ariya/phantomjs#12750 screenshot tests that often fail randomly are:
|
This was referenced Sep 25, 2016
Note to self: would it be maybe possible to "Retry on failure" for those random UI test failures? Maybe there's a feature in moccha to retry failed tests... |
mattab
added a commit
that referenced
this issue
Sep 28, 2016
mattab
added a commit
that referenced
this issue
Sep 29, 2016
* Fix depraction test: use assertDeprecatedMethodIsRemovedInPiwik3 * Fix Scheduled Reports sent one hour late in daylight saving timezones (#10443) * convert hour to send report to/from UTC, to ensure it isn't affected by daylight savings * adds update script to change existing scheduled reports to use utc time * code improvement * adds missing param * Added new event Archiving.makeNewArchiverObject to allow customising plugin archiving (#10366) * added hook to alllow plugin archiving prevention * cr code style notes * reworked PR to fit CR suggestions * added PHPDoc for hook * Event description more consistent * UI tests: minor changes * Comment out Visitor Log UI tests refs #10536 * Adds test checking if all screenshots are stored in lfs * removed screenshots not stored in lfs * readds screenshots to lfs * 2.16.3-b4 * Issue translation updates against 2.x-dev * language update * Fix bug in widget list remove where the JSON object becomes array * 2.16.3-rc1 * console command custom-piwik-js:update should work when directory is writable and file does not exist yet (#10576) * followup #10449 * Fix test (cherry picked from commit fac3d63) * Prevent chmod(): No such file or directory * Automatically update all marketplace plugins when updating Piwik (#10527) * update plugins and piwik at the same time * make sure plugins are updated with piwik * use only one try/catch * reload plugin information once it has been installed * make sure to clear caches after an update * fix ui tests * make sure to use correct php version without any extras * Additional informations passed in the hook "isExcludedVisit" (issue #10415) (#10564) * Additional informations passed in the hook "isExcludedVisit" (issue #10415) * Added better description to the new parameters * Update VisitExcluded.php * Remove two parameters not needed as better to use the Request object * Update VisitExcluded.php * remove extra two parameters in VisitExcluded constructor to prevent confusion (#10593) * Update our libs to latest #10526 * Update composer libraries to latest #10526 * Update log analytics to latest * When updating the config file failed (or when any other file is not writable...), the Updater (for core or plugins) will now automatically throw an error and cancel the update (#10423) * When updating the config file failed (or when any other file is not writable...), the Updater (for core or plugins) will now automatically throw an error and cancel the update * add integration test to check the correct exception is thrown when config not writabel * New integration test for updater * Make test better * When opening the visitor profile, do not apply the segment (#10533) * When opening the visitor profile, do not apply the segment * added ui test for profile but does work for me * next try to make ui test work * add expected screenshot * added missing doc
mattab
added a commit
that referenced
this issue
Sep 30, 2016
This was referenced Oct 4, 2016
We've implemented retry on failure using Mocha retry, and it works well it seems |
mattab
added
the
answered
For when a question was asked and we referred to forum or answered it.
label
Feb 28, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
answered
For when a question was asked and we referred to forum or answered it.
c: Tests & QA
For issues related to automated tests or making it easier to QA & test issues.
Often the visitor log UI tests, and overlay UI tests, are randomly failing. as we need to have a green build for UI tests, for now we must disable these tests. It's bad practise to disable UI tests but we have no choice if we want to see a green build for other practical reasons.
Ideally in the future phantomjs2 will find the solution to this random failure and we can re-enable these UI tests.
The text was updated successfully, but these errors were encountered: