Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Piwik 3.2 : Memory / time exhaustion in visitor log #12188

Closed
ArnY opened this issue Oct 13, 2017 · 4 comments
Closed

Piwik 3.2 : Memory / time exhaustion in visitor log #12188

ArnY opened this issue Oct 13, 2017 · 4 comments
Labels
duplicate For issues that already existed in our issue tracker and were reported previously.

Comments

@ArnY
Copy link
Contributor

ArnY commented Oct 13, 2017

I just upgraded my piwik instance from 3.0.3 to 3.2. Since the upgrade, my users are unable to access the visitor log since it requires so much memory it hits the limits no matter the selected range. Even a single day of visitor log consumes up to 4Gb of memory or hits the 600s time limit.

I had no such problem with piwik 3.0.3.

Example of error:

Allowed memory size of 2147483648 bytes exhausted (tried to allocate 132122504 bytes) in /var/www/piwik/core/DataTable/Renderer/Html.php on line 182, referer: https://analytics.univ-nantes.fr/index.php?module=CoreHome&action=index&widget=undefined&idSite=9&period=range&date=previous7&showtitle=1&random=9570&updated=2

Tried with a 4G limit, but that wasn't enough either.

I noticed memory exhaustion errors in several scripts:
/var/www/piwik/libs/Zend/Db/Statement/Pdo.php
/var/www/piwik/plugins/CorePluginsAdmin/Controller.php
/var/www/piwik/core/AssetManager/UIAssetCacheBuster.php
/var/www/piwik/core/AssetManager/UIAsset/OnDiskUIAsset.php
/var/www/piwik/core/AssetManager/UIAsset/OnDiskUIAsset.php

Piwik: 3.2.0
MySQL: 5.5.5
PHP: 7.0.19-1 (tried with php 5.6 also)

@ArnY
Copy link
Contributor Author

ArnY commented Oct 19, 2017

I finally got a visitor log for our busiest site: sunday october 1st, sundays are our quietest days. Summary says:

117 817 visits, 14 254 single visitors

not much, but I had to pump up the memory limit up to 8Gb and it took almost 7 minutes for my browser to be able to display the page (not loading it, just displaying it). Once loaded it appears to be the whole day data, without any pagination, which could explain why the page took so much memory and time to display.

Very very strange change of behavior since Piwik 3.0.3 with which I could browse visitor log reports without any problem. Could it be a change in the pagination system or something like that that would explain it now seems to be gathering and display all the data in a single page?

Could it be not related to piwik and rather to my system instead? A few hours before upgrading piwik I upgraded the server from debian 8 to debian 9. From php 5.6 to php 7.0, from mysql 5.5 to mariaDB 10.1. Also, I enabled the QueuedTracking plugin and configured redis a few days before the upgrade.

@mattab
Copy link
Member

mattab commented Oct 19, 2017

Hi @ArnY

117 817 visits, 14 254 single visitors
not much, but I had to pump up the memory limit up to 8Gb and it took almost 7 minutes for my browser to be able to display the page (not loading it, just displaying it).

Did you select "all" in the bottom right selector, and all 117K visits were loaded? or do you only load like 500?

Could it be not related to piwik and rather to my system instead?

Yes, it could be, we changed many things in Visitor log / visitor profile in 3.1.0 in #6111

@ArnY
Copy link
Contributor Author

ArnY commented Oct 21, 2017

Hrm, the pager was indeed set to "all" that explains why there was no pagination. My bad, one of my users must have changed that when the visits were still very low. Is there a way to enforce a pagination setting like 500 by default instead of keeping the last user's choice? It took my a very long time to be able to change it since no matter which day I picked the visitor log was always too heavy for the browser to handle.

@mattab
Copy link
Member

mattab commented Sep 25, 2018

Btw we fixed this problem forever by removing the "All" option in the report. 👍

@mattab mattab closed this as completed Sep 25, 2018
@mattab mattab added the duplicate For issues that already existed in our issue tracker and were reported previously. label Sep 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate For issues that already existed in our issue tracker and were reported previously.
Projects
None yet
Development

No branches or pull requests

2 participants