Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Archiving fails when period is year and date is last3 #15284

Closed
toredash opened this issue Dec 18, 2019 · 4 comments
Closed

Archiving fails when period is year and date is last3 #15284

toredash opened this issue Dec 18, 2019 · 4 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@toredash
Copy link
Contributor

There seems to be an issue with archiving in 3.13.0.

Archiving i started as documented:
/var/www/console core:archive -vvv

We have noticed that for one site that archiving fails when period=year&date=last3:

INFO [2019-12-18 07:46:53] 24567  Error: Got invalid response from API request: ?module=API&method=API.get&idSite=8&period=year&date=last3&format=php&trigger=archivephp.
ERROR [2019-12-18 07:46:53] 24567 1 total errors during this script execution, please investigate and try and fix these errors.
[Exception]
1 total errors during this script execution, please investigate and try and fix these errors.
Exception trace:
() at /var/www/core/CronArchive.php:589
Piwik\CronArchive->logFatalError() at /var/www/core/CronArchive.php:582
Piwik\CronArchive->end() at /var/www/core/CronArchive.php:326
Piwik\CronArchive->Piwik\{closure}() at /var/www/core/Access.php:639
Piwik\Access::doAsSuperUser() at /var/www/core/CronArchive.php:327
Piwik\CronArchive->main() at /var/www/plugins/CoreConsole/Commands/CoreArchiver.php:27
Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute() at /var/www/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /var/www/core/Console.php:103
Piwik\Console->Piwik\{closure}() at /var/www/core/Access.php:639
Piwik\Access::doAsSuperUser() at /var/www/core/Console.php:104
Piwik\Console->doRun() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:126
2019-12-18T08:46:53.780+01:00
Symfony\Component\Console\Application->run() at /var/www/console:27

Related issue: #15086

For all other sites, a date value of last2 is used. But for this one site, a value of last3 is used for week, month, year. The day periode is using last2:

image

Note that it is only period=year&date=last3 that fails, period=[week,month]&date=last3 executes successfully.

I think the problem is in this function:

matomo/core/CronArchive.php

Line 1627 in ee43c6a

private function getDateLastN($idSite, $period, $lastTimestampWebsiteProcessed)

Our current workaround is to start the archive process with --force-date-last-n 2
@tsteur
Copy link
Member

tsteur commented Dec 18, 2019

@toredash last3 shouldn't cause any issue if the other years are already archived? How does it fail? are you running out of memory?

@toredash
Copy link
Contributor Author

@tsteur You are correct, this is not a bug. We are in fact running out of memory but it isn't reported anywhere.

In my env we use AWS ECS to run container instances of Matomo. There is hard and soft memory limits set on the process that performs the archive process. When it runs out of memory and swap, it seems docker kills the offending php process, not the docker container itself. This contradicts what I'm reading from the docker docs, but I'm not going to spend more time on trying to figure out why.

This is probably why the response in matomo isn't clearer as well, the process is killed and no return data is given. Could that explain why the error msg isn't more clear, or is this as good as it gets ?

@tsteur tsteur added the answered For when a question was asked and we referred to forum or answered it. label Dec 19, 2019
@tsteur
Copy link
Member

tsteur commented Dec 19, 2019

Yes, when there is a memory issue we can't really do much about it unfortunately. I'll close the issue if that's alright with you?

@tsteur tsteur closed this as completed Dec 19, 2019
@toredash
Copy link
Contributor Author

Of course!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tsteur @toredash