Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Piwik Archiving Process is eating up ram #10194

Closed
FarhadF opened this issue May 30, 2016 · 5 comments
Closed

Piwik Archiving Process is eating up ram #10194

FarhadF opened this issue May 30, 2016 · 5 comments
Labels
answered For when a question was asked and we referred to forum or answered it. c: Performance For when we could improve the performance / speed of Matomo. Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users.
Milestone

Comments

@FarhadF
Copy link

FarhadF commented May 30, 2016

Hi guys,

I am using Piwik in my production environment and everything is working fine. But archiving process is demanding more and more memory. Here is screenshot of the error.
I have currently set 7GB Memroy Limit in php.ini. Is this behavior normal?

l_41b tmp

any help would be appreciated
Regards,
Farhad

@tsteur
Copy link
Member

tsteur commented Jun 2, 2016

Do you know how many actions there were in the last year?

Can you maybe execute select count(*), count(distinct idaction_url_ref), count(distinct idaction_name_ref), count(distinct idvisit), count(distinct idvisitor) from piwik_log_link_visit_action where server_time like '2016%' and idsite = 1;

@FarhadF
Copy link
Author

FarhadF commented Jun 5, 2016

Hi,
thanks for the reply, Here are the results.
image
any help would be appreciated
Regards,
Farhad

@tsteur
Copy link
Member

tsteur commented Jun 5, 2016

For 28 mio actions it shouldn't take up that much memory I reckon. Maybe having so many different URLs is a problem that lead to more memory consumption. I wonder if it maybe helps to reduce the number of rows in action tables see http://piwik.org/faq/how-to/faq_54/

The default value for datatable_archiving_maximum_rows_actions is currently 500 and maybe it gets better if you set it to like 300. It might not fix the problem though as the other reports for the past months are already archived. It may be needed to invalidate previously archived month reports and archive them again which might actually not take too long. I can give you some instructions for this. Can you try if it already helps by just setting the config value to eg 300?

Are you using the latest Piwik 2.16.1 version btw? I currently don't have access to a server with lots of data so I cannot really profile anything currently. Also profiling wouldn't help much as memory consumption is different depending on the metrics above. I presume the problem here is having too many different distinct URLs and action names. Any idea maybe @mattab ?

@mattab
Copy link
Member

mattab commented Jul 7, 2016

please check you're using latest Piwik version and latest PHP version. After upgrade, do you still have this memory issue? @FarhadF

@mattab mattab added c: Performance For when we could improve the performance / speed of Matomo. Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users. labels Jul 14, 2016
@mattab mattab added this to the 2.16.x (LTS) milestone Jul 14, 2016
@mattab
Copy link
Member

mattab commented Aug 23, 2016

As we have not heard from you, closing the issue. Feel free to comment and re-open if you still experience the issue and have more info.

@mattab mattab closed this as completed Aug 23, 2016
@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label Aug 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it. c: Performance For when we could improve the performance / speed of Matomo. Waiting for user feedback Indicates the Matomo team is waiting for feedback from the author or other users.
Projects
None yet
Development

No branches or pull requests

3 participants