Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests are failing on PHP 7.4 and above #17721

Closed
2 tasks
sgiehl opened this issue Jun 29, 2021 · 4 comments · Fixed by #17852
Closed
2 tasks

Tests are failing on PHP 7.4 and above #17721

sgiehl opened this issue Jun 29, 2021 · 4 comments · Fixed by #17852
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone

Comments

@sgiehl
Copy link
Member

sgiehl commented Jun 29, 2021

Currently our AllTests suite is silently failing on TravisCI. See https://travis-ci.com/github/matomo-org/matomo/jobs/520002593#L792
The test that silently aborts is this one:

public function test_UpdateCommand_ReturnsCorrectExitCode_WhenErrorOccurs()
{
// create a blob table, then drop it manually so update 2.10.0-b10 will fail
$tableName = ArchiveTableCreator::getBlobTable(Date::factory('2015-01-01'));
Db::exec("DROP TABLE $tableName");
$result = $this->applicationTester->run(array(
'command' => 'core:update',
'--yes' => true
));
$this->assertEquals(1, $result, $this->getCommandDisplayOutputErrorMessage());
self::assertStringContainsString("Matomo could not be updated! See above for more information.", $this->applicationTester->getDisplay());
}

Already saw the same behavior while trying to fix the tests for PHP 8 in #16897

When running the UpdateTests locally on PHP 7.2, 7.4 or 8.0 it works without problems for me.

@matomo-org/core-reviewers is anyone able to reproduce the test failure locally?

Target of this issue should be to

  • Check why the build is actually green even if the test aborts
  • Check why the test fails and fix it
@sgiehl sgiehl added c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. Regression Indicates a feature used to work in a certain way but it no longer does even though it should. labels Jun 29, 2021
@sgiehl sgiehl added this to the 4.4.0 milestone Jun 29, 2021
@diosmosis
Copy link
Member

Looks like an error occurs on mysqli (in travis at least):

Array
(
    [warnings] => Array
        (
        )

    [errors] => Array
        (
            [0] => Mysqli prepare error: Table 'piwik_tests.archive_blob_2015_01' doesn't exist
        )

    [coreError] => 1

    [deactivatedPlugins] => Array
        (
        )
)

This gets thrown as a runtime exception which I guess stops the test entirely.

@sgiehl
Copy link
Member Author

sgiehl commented Jul 1, 2021

This error is caused by the test on purpose. Not sure why it's a runtime exception in this case and why it works locally with MYSQLI 🤔

@diosmosis
Copy link
Member

Haven't figured out the cause, but:

  • replacing the runtime exception w/ \Exception has no effect
  • and catching the exception in the Update command allows the test to continue

@diosmosis
Copy link
Member

I've narrowed down the problem to something in the Application.php class in symfony console. The run() method has a try-catch, the catch is never entered, the process seems to just exit. However, in doRun() if I surround doRunCommand() and catch the exception, the tests continue. So I suspect the issue is Console::doRun()'s safemode logic.

@mattab mattab modified the milestones: 4.4.0, 4.5.0 Jul 28, 2021
@sgiehl sgiehl linked a pull request Aug 5, 2021 that will close this issue
10 tasks
@justinvelluppillai justinvelluppillai added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Oct 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Tests & QA For issues related to automated tests or making it easier to QA & test issues. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants