New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Omnifixture Update for automated UI screenshot tests #9272
Conversation
Note: The rowevolution screenshots need #9241 to be merged in order to be processed correct. I'll do a rebase after that is done. |
My approach to updating OmniFixture is to ignore changes that are just to data. It would be near impossible to isolate any data inaccuracies in the UI tests, IMO it's better to rely on system tests for that. If the new data triggers UI bugs (ie, css issues, double encoding, xss issues, etc.), then it should be fixed, but otherwise trying to check if the data is accurate would only be waste of time. |
maybe you have some idea why the omnifixture now has less data @sgiehl @diosmosis ? |
6807025
to
ce29fbc
Compare
121c13c
to
8e993b4
Compare
1a71d15
to
ea80bac
Compare
ea80bac
to
5309b38
Compare
371d5bb
to
64b0244
Compare
I've tested for a while where the data differences are coming from. Does anyone know if there were any changes in the past few months that could cause the OmniFixture not to contain full archives? |
Feel free to work on it.
You may have a look at the two fixes I had commited on this branch:
The first fixes builing the Omnifixture, which was broken for me again
after rebasing on master, the second fixes the Ui tests for switching the
display types...
|
I've rebased my local changes w/ your remote changes, so they're included for me locally. |
45b73cc
to
24a2360
Compare
@mattab Latest build may pass, you can see the screenshot changes here: https://github.com/piwik/piwik-ui-tests/pull/5/files |
@diosmosis Awesome! 👍 Could you please resolve conflict and merge both PR? |
…nger needed so when fixtures are setup together, the change won't carry over into other fixtures.
…adRealTranslations=1 is needed for archiving) so results are accurate. Also, do not use Fixture::getTestEnvironment() in fixtures, instead create new instances of TestingEnvironmentVariables so changes made to the file are not ignored/lost.
…PivotByDimension tests.
…r initial values before setting up each initial fixture. Ensures that changes to site properties by fixtures are reset before setting up the next fixture.
… so excessive sanitization is used and site name is stored properly in OmniFixture DB.
…cted screenshots submodule.
010ade2
to
73d8af0
Compare
Omnifixture wasn't update for a while now, so there are a lot of changes in UI tests that needs to be checked. Could anyone (@mattab, @tsteur, @diosmosis) please check the changed UI screenshots and mark the changes as correct or not.