@github-actions[bot] opened this Pull Request on August 19th 2022 Contributor

Updated submodules:

  • plugins/TagManager
@sgiehl commented on August 19th 2022 Member

It took me a couple of hours to figure out, why the latest updates in TagManager let our UITestFixture start failing to set up.
This was caused by adding a update script to the plugin. Due to the way we used to set up the fixture, the update script wasn't executed, causing the database being out of sync. Found a proper solution, which should fix it. Might be good if someone could have a quick look at the test fix.

@bx80 commented on August 21st 2022 Contributor

There seem to be a lot of UI tests failing, I can't see any good reason for that based on the change, so I've trigger another test build in case there was a transient issue with the test environment.

@sgiehl commented on August 22nd 2022 Member

I really don't have a clue why the tests start failing with the changes I've added. Tried it in a couple of different ways now, but all seem to produce some unrelated failures in the UI tests. The basic difference now, is that the updater now also executes the update scripts of plugins. In that case this are AnonymousPiwikUsageMeasurement and TagManager. But running those scripts shouldn't produce such failures.
On my local dev machine setting up the fixture works and the tests failing on travis are also passing (at least the ones I tried).

I already spent too much time trying to find a quick fix solution here, so I'll let you (@justinvelluppillai) decide how to proceed with this:

  • keep trying to find a fix for the current code by debugging for an unpredictable amount of time
  • update the ui test fixture, which might easily take up to two days, as it wasn't done since Matomo 3.16 and thus will take some time to validate all the changes...

another option would be to fully refactor how our ui tests work, but that would be out of scope for sure, as we can't update the TagManager submodule till this was fixed and did not yet have a discussion how our UI tests could be handled in the future.

@justinvelluppillai commented on August 22nd 2022 Member

@sgiehl yes we'll need to put some more time in to figure this out. It seems like some of the fixtures aren't loaded in the same way or something? I'll see if @peterhashair has the same issues with the github action tests runner, after that we can discuss again.

@peterhashair commented on August 22nd 2022 Contributor

merge this with GitHub Action tests here https://github.com/matomo-org/matomo/pull/19659 @sgiehl seems like the Github Action doesn't have the UI failed, but not too sure if that's right.

This Pull Request was closed on August 26th 2022
Powered by GitHub Issue Mirror