In most cases the build is currently failing as the system tests for mysqli get terminated after 40min.
For testing purpose I added a small script to check the running times of tests > 5 sec:
https://travis-ci.org/matomo-org/matomo/jobs/360648037
Tests running very long are:
Piwik\Tests\System\ArchiveCronTest::testArchivePhpCron
took 438.5 secondsPiwik\Tests\System\ArchiveWebTest::test_WebArchiveScriptCanBeRun_WithPhpCgi_AndWithoutTokenAuth
took 466.8 secondsSetup of the fixture MaySitesImportedLogs
only takes ~30 sec, but contains a lot of data. Maybe we should use a smaller fixture for those tests to run faster?
And some of the API tests Piwik\Tests\System\AutoSuggestAPITest::testAnotherApi
take alomst 60sec
ArchiveWebTest moved to Integration tests for now, so tests are passing again. Nevertheless we should check if we can improve the Archiving tests to run faster...
As found by Phpunit speedtrap in https://github.com/matomo-org/matomo/pull/12763#issuecomment-383531452 the current slowest tests are:
You should really fix these slow tests (>500ms)...
1. 215204ms to run Piwik\Tests\System\ArchiveCronTest:testArchivePhpCron
2. 59605ms to run Piwik\Tests\System\TwoVisitorsTwoWebsitesDifferentDaysTest:testApi with data set <a href='/7'>#7</a>
3. 54062ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/92'>#92</a>
4. 51416ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/96'>#96</a>
5. 51372ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/90'>#90</a>
6. 50215ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/74'>#74</a>
7. 48881ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/72'>#72</a>
8. 48671ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/94'>#94</a>
9. 47777ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/66'>#66</a>
10. 47513ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set <a href='/58'>#58</a>
...and there are 202 more above your threshold hidden from view
Time: 32.84 minutes, Memory: 368.00MB
@sgiehl this issue was probably fixed a while ago, but noticed now the IntegrationTest suite is timing out (running through 99% but timing out at 40min) eg. https://travis-ci.org/matomo-org/matomo/jobs/547052031
I have checked for long running tests on travis. Actually there are currently only 2 tests that take more than 10 seconds:
Moving this for now to priority backlog. I reckon the tests aren't really a problem now. Since we made a lot of changes recently would eventually need to look again at which tests take a long time and then see what we can do. Had a look at the 2 tests that take long. The static file that is not readable not sure why it takes so long. I wonder if it was maybe only once taking long by "accident". The other one is fetching quite a few reports over quite a few days which explains why it takes maybe 13s (which maybe isn't all that long). It's possible it's faster now.
Need to see again later when it becomes an issue again.
@tsteur the static file test should already be fast now with this change: https://github.com/matomo-org/matomo/pull/14605/files