Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping outdated archives can time out on huge Piwik instances #5354

Closed
mgazdzik opened this issue Jun 17, 2014 · 1 comment
Closed

Dropping outdated archives can time out on huge Piwik instances #5354

mgazdzik opened this issue Jun 17, 2014 · 1 comment
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc.

Comments

@mgazdzik
Copy link
Contributor

There is certain problem arising when daily statistics have many segments and/or many idsites. In file https://github.com/piwik/piwik/blob/master/core/DataAccess/ArchiveSelector.php#L317

there ocassionaly is being triggered huge DELETE query with up to 20k elements in "WHERE IN" clause. This causes server to drop connection to DB server.

Maybe it could be fixed by introducing new config variable allowing to define sharding limit for this query. For ex. it could be [General] purging_outdated_archives_limit = 0 . By default it would be set to 0 - then no sharding will be applied (for small/medium instances where this does not cause problems). After changing to bigger value - single query should be replaced with iteration of queries with X idarchives in one batch executed for all necessary idarchives.

@mattab
Copy link
Member

mattab commented Jun 18, 2014

In 4361b23: fixes #5354 Delete archives by batch of 1000 using array_chunk

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

2 participants