@JRPonte opened this Issue on August 31st 2022

Hi all,
Since I updated my installation of matomo to version 4.11, the archive process of one of the sites that matomo is tracking has been systematically giving "Allowed memory exhausted" errors.
I've already increased PHP's memory to 8GB, start servers are 50, max children 100, and max requests 5000.
I've disabled archiving through the browser, and I've one cron job that runs every 5 minutes.
Matomo currently logs 35 sites and an average os 750k pageviews per day.

If the errors are limited to only 1 site, how can it be corrected? Should all archives be invalidated so that matomo can run the archive process again?

Matomo Version: 4.11.0
PHP Version: 7.3.28
MySQL: 10.3.29-MariaDB
Server OS: Debian 10.12
Server: Azure's standard D4as v4 (4 vcpus, 16 GiB memory)
CPU usage is normally at 90%+ during office hours, and drops to 30%+ outside that

Error log:
Error unserializing the following response from ?module=API&method=CoreAdminHome.archiveReports&idSite=34&period=day&date=2022-08-18&format=json&trigger=archivephp: 'PHP Fatal error: Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) in /opt/bitnami/matomo/libs/Zend/Db/Statement/Mysqli.php on line 250 {"result":"error","message":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\Plugins\CorePluginsAdmin\Controller->safemode(Array)\n#1 \/opt\/bitnami\/matomo\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/matomo\/core\/FrontController.php(169): Piwik\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/matomo\/core\/FrontController.php(100): Piwik\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/matomo\/core\/FrontController.php(275): Piwik\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\FrontController::triggerSafeModeWhenError()\n#6 {main}"}{"result":"error","message":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\Plugins\CorePluginsAdmin\Controller->safemode(Array)\n#1 \/opt\/bitnami\/matomo\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/matomo\/core\/FrontController.php(169): Piwik\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/matomo\/core\/FrontController.php(100): Piwik\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/matomo\/core\/FrontController.php(275): Piwik\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\FrontController::triggerSafeModeWhenError()\n#6 {main}"}'

Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=34&period=day&date=2022-08-18&format=json&trigger=archivephp. Response was 'PHP Fatal error: Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) in /opt/bitnami/matomo/libs/Zend/Db/Statement/Mysqli.php on line 250 {"result":"error","message":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\Plugins\CorePluginsAdmin\Controller->safemode(Array)\n#1 \/opt\/bitnami\/matomo\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/matomo\/core\/FrontController.php(169): Piwik\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/matomo\/core\/FrontController.php(100): Piwik\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/matomo\/core\/FrontController.php(275): Piwik\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\FrontController::triggerSafeModeWhenError()\n#6 {main}"}{"result":"error","message":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 8588886016 bytes exhausted (tried to allocate 7627087104 bytes) on \/opt\/bitnami\/matomo\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/matomo\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\Plugins\Actions\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\Plugins\CorePluginsAdmin\Controller->safemode(Array)\n#1 \/opt\/bitnami\/matomo\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/matomo\/core\/FrontController.php(169): Piwik\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/matomo\/core\/FrontController.php(100): Piwik\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/matomo\/core\/FrontController.php(275): Piwik\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\FrontController::triggerSafeModeWhenError()\n#6 {main}"}'

@mikkeschiren commented on August 31st 2022 Contributor

We also see much more need for RAM since upgrading our setups, with a lot of traffic.

@sgiehl commented on September 2nd 2022 Member

I assume this might most likely be caused by this new feature: https://github.com/matomo-org/matomo/pull/18221

@JRPonte commented on September 3rd 2022

Can I do something to prevent this errors? Or should I just wait for the next update?

@OlliWu commented on September 5th 2022

Just want to mention that we're also facing the same issue. We're tracking a few sites with lots of traffic. But only the one with a handfull of Events and Goals defined using a huge amount of Memory during archiving. The Process have peaks of 15-20 GB RAM depending on segment and period. We tweaked all necessary parameters to prevent errors. But monthly reports are still likely to fail.

@sgiehl commented on September 5th 2022 Member

As mentioned above, this is most likely caused by https://github.com/matomo-org/matomo/pull/18221.
We are aware that archiving consumes a lot more memory with that. We are currently evaluating possible improvements.
Maybe @bx80 has more insights on that...

@djmaze commented on September 16th 2022

Since upgrading, we are also not able to successfully run archive jobs on our server because of a hugely increased memory usage.

Is it possible to "disable" the new feature somehow? Any other workaround?

@bx80 commented on September 20th 2022 Contributor

Sorry about the delay with this. The next release (currently in release candidate status) will include a config option to disable archiving of action goal conversion data. We're looking at options to improve performance and reduce memory usage for this feature on larger datasets.

Powered by GitHub Issue Mirror