@diosmosis opened this Pull Request on June 10th 2018 Member

Work in progress, more to follow.

@fdellwing commented on June 21st 2018 Contributor

Will this PR make #13036 work in tests?

@diosmosis commented on June 21st 2018 Member

Commented on the other issue. This PR probably won't be finished for a while unfortunately :)

@sgiehl commented on June 24th 2018 Member

@diosmosis do you need help finishing this PR? Could maybe help converting some tests when I have some time in between...

@diosmosis commented on June 24th 2018 Member

I was actually doing this in my spare time but it took longer than expected (was hoping to finish in a weekend), and now I don't have the spare time. Would be happy for the help, feel free to push directly to the branch if you'd like @sgiehl!

Note: in order to run the tests, you should just have to do an npm install (w/ node 8 or above) in the tests/lib/screenshot-testing dir, then just run ./console tests:run ... as usual. Still need to have the right fonts & everything. On a mac I can only run them locally through docker (well, run them & have the tests pass).

@sgiehl commented on July 17th 2018 Member

@diosmosis I've rebased the branch on latest 3.x-dev and added more conversions. Most core tests are now converted, but there are some tests failing due to failing requests and I couldn't find out why. I've added the last commit to fix some failing requests for the logos, but that one should be removed again as soon as we know why the requests were failing in the first place. Maybe you have an idea.

@sgiehl commented on January 5th 2019 Member

@diosmosis I've rebased and update this branch with latest changes on 3.x-dev. ~90% of tests are already passing, but there's still some work left. Maybe you have some time in between to work on some of the stuff:

  • Installation, Login & TwoFactorAuth tests are currently skipped in a11d7bbd190905ad6a7385292ebf4ce6b94ddeaa as puppeteer completely hangs (rans into travis 10min timeout if returns are removed)
  • I've disabled the dangerous link check in 2fd9369a50732f428f2a3959bc9521a328599829 as it does not yet work again
  • Diffs aren't uploaded to artifacts server, and thus the failures aren't shown there
  • Trackingfailures tests don't seem to be able to generate the failures anymore
  • TagManager tests are not yet done
  • Some tests are still randomly failing as they slightly differ from time to time. Currently only a difference of 1 pixel is allowed, maybe we need to reimplement the comparison threshold.
@diosmosis commented on January 5th 2019 Member

👍 will make some time for it. It's very annoying to have UI tests that never consistently pass, hope this can be merged soon.

@diosmosis commented on January 16th 2019 Member

The installation test failure is due to https://github.com/GoogleChrome/puppeteer/issues/3471. W/o request interception it works. There was a recent commit in puppeteer to try and resolve this, but it doesn't seem to help in this case.

Powered by GitHub Issue Mirror