Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some tests on travis are running too long #12691

Closed
sgiehl opened this issue Apr 1, 2018 · 7 comments
Closed

Some tests on travis are running too long #12691

sgiehl opened this issue Apr 1, 2018 · 7 comments
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues.
Milestone
Backlog (Help wan...

Comments

@sgiehl
Copy link
Member

sgiehl commented Apr 1, 2018

In most cases the build is currently failing as the system tests for mysqli get terminated after 40min.
For testing purpose I added a small script to check the running times of tests > 5 sec:

https://travis-ci.org/matomo-org/matomo/jobs/360648037

Tests running very long are:

  • Test Piwik\Tests\System\ArchiveCronTest::testArchivePhpCron took 438.5 seconds
  • ​​​​​​​​Test Piwik\Tests\System\ArchiveWebTest::test_WebArchiveScriptCanBeRun_WithPhpCgi_AndWithoutTokenAuth took 466.8 seconds

Setup of the fixture MaySitesImportedLogs only takes ~30 sec, but contains a lot of data. Maybe we should use a smaller fixture for those tests to run faster?

And some of the API tests Piwik\Tests\System\AutoSuggestAPITest::testAnotherApi take alomst 60sec
@sgiehl sgiehl added the c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. label Apr 1, 2018
@sgiehl sgiehl mentioned this issue Apr 1, 2018
Merged
@sgiehl
Copy link
Member Author

sgiehl commented Apr 1, 2018

ArchiveWebTest moved to Integration tests for now, so tests are passing again. Nevertheless we should check if we can improve the Archiving tests to run faster...

@mattab
Copy link
Member

mattab commented Jun 26, 2018

As found by Phpunit speedtrap in #12763 (comment) the current slowest tests are:

You should really fix these slow tests (>500ms)...
 1. 215204ms to run Piwik\Tests\System\ArchiveCronTest:testArchivePhpCron
 2. 59605ms to run Piwik\Tests\System\TwoVisitorsTwoWebsitesDifferentDaysTest:testApi with data set #7
 3. 54062ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #92
 4. 51416ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #96
 5. 51372ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #90
 6. 50215ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #74
 7. 48881ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #72
 8. 48671ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #94
 9. 47777ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #66
 10. 47513ms to run Piwik\Tests\System\AutoSuggestAPITest:testAnotherApi with data set #58
...and there are 202 more above your threshold hidden from view
Time: 32.84 minutes, Memory: 368.00MB

@mattab mattab mentioned this issue Jun 26, 2018
Closed
@mattab
Copy link
Member

mattab commented Jun 18, 2019

@sgiehl this issue was probably fixed a while ago, but noticed now the IntegrationTest suite is timing out (running through 99% but timing out at 40min) eg. https://travis-ci.org/matomo-org/matomo/jobs/547052031

@mattab mattab changed the title System tests on travis with MYSQLI are running too long Some tests on travis are running too long Jun 18, 2019
@mattab mattab added this to the 3.11.0 milestone Jun 18, 2019
@sgiehl
Copy link
Member Author

sgiehl commented Jul 1, 2019

I have checked for long running tests on travis. Actually there are currently only 2 tests that take more than 10 seconds:

  • 13856ms to run Piwik\Plugins\PrivacyManager\tests\Integration\DataPurgingTest:test_LogDataPurging_WorksWhenVisitsInPastTracked
  • 65079ms to run Piwik\Tests\Integration\ServeStaticFileTest:test_nonReadableFile

@sgiehl sgiehl mentioned this issue Jul 1, 2019
Merged
@sgiehl sgiehl mentioned this issue Jul 8, 2019
Merged
@mattab mattab modified the milestones: 3.11.0, 3.12.0 Jul 23, 2019
@tsteur
Copy link
Member

tsteur commented Sep 3, 2019

Moving this for now to priority backlog. I reckon the tests aren't really a problem now. Since we made a lot of changes recently would eventually need to look again at which tests take a long time and then see what we can do. Had a look at the 2 tests that take long. The static file that is not readable not sure why it takes so long. I wonder if it was maybe only once taking long by "accident". The other one is fetching quite a few reports over quite a few days which explains why it takes maybe 13s (which maybe isn't all that long). It's possible it's faster now.
Need to see again later when it becomes an issue again.

@sgiehl
Copy link
Member Author

sgiehl commented Sep 4, 2019

@tsteur the static file test should already be fast now with this change: https://github.com/matomo-org/matomo/pull/14605/files

@tsteur
Copy link
Member

tsteur commented Sep 4, 2019

Closing the issue for now 👍

@tsteur tsteur closed this as completed Sep 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@tsteur @mattab @sgiehl