@toredash opened this Issue on December 18th 2019 Contributor

There seems to be an issue with archiving in 3.13.0.

Archiving i started as documented:
/var/www/console core:archive -vvv

We have noticed that for one site that archiving fails when period=year&date=last3:

INFO [2019-12-18 07:46:53] 24567  Error: Got invalid response from API request: ?module=API&method=API.get&idSite=8&period=year&date=last3&format=php&trigger=archivephp.
ERROR [2019-12-18 07:46:53] 24567 1 total errors during this script execution, please investigate and try and fix these errors.
[Exception]
1 total errors during this script execution, please investigate and try and fix these errors.
Exception trace:
() at /var/www/core/CronArchive.php:589
Piwik\CronArchive->logFatalError() at /var/www/core/CronArchive.php:582
Piwik\CronArchive->end() at /var/www/core/CronArchive.php:326
Piwik\CronArchive->Piwik\{closure}() at /var/www/core/Access.php:639
Piwik\Access::doAsSuperUser() at /var/www/core/CronArchive.php:327
Piwik\CronArchive->main() at /var/www/plugins/CoreConsole/Commands/CoreArchiver.php:27
Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute() at /var/www/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /var/www/core/Console.php:103
Piwik\Console->Piwik\{closure}() at /var/www/core/Access.php:639
Piwik\Access::doAsSuperUser() at /var/www/core/Console.php:104
Piwik\Console->doRun() at /var/www/vendor/symfony/console/Symfony/Component/Console/Application.php:126
2019-12-18T08:46:53.780+01:00
Symfony\Component\Console\Application->run() at /var/www/console:27

Related issue: #15086

For all other sites, a date value of last2 is used. But for this one site, a value of last3 is used for week, month, year. The day periode is using last2:

image

Note that it is only period=year&date=last3 that fails, period=[week,month]&date=last3 executes successfully.

I think the problem is in this function: https://github.com/matomo-org/matomo/blob/ee43c6a674e117db47f1f19060316300c5935006/core/CronArchive.php#L1627

Our current workaround is to start the archive process with --force-date-last-n 2

@tsteur commented on December 18th 2019 Member

@toredash last3 shouldn't cause any issue if the other years are already archived? How does it fail? are you running out of memory?

@toredash commented on December 19th 2019 Contributor

@tsteur You are correct, this is not a bug. We are in fact running out of memory but it isn't reported anywhere.

In my env we use AWS ECS to run container instances of Matomo. There is hard and soft memory limits set on the process that performs the archive process. When it runs out of memory and swap, it seems docker kills the offending php process, not the docker container itself. This contradicts what I'm reading from the docker docs, but I'm not going to spend more time on trying to figure out why.

This is probably why the response in matomo isn't clearer as well, the process is killed and no return data is given. Could that explain why the error msg isn't more clear, or is this as good as it gets ?

@tsteur commented on December 19th 2019 Member

Yes, when there is a memory issue we can't really do much about it unfortunately. I'll close the issue if that's alright with you?

@toredash commented on December 20th 2019 Contributor

Of course!

This Issue was closed on December 19th 2019
Powered by GitHub Issue Mirror