@paladox opened this Issue on August 31st 2019 Contributor

Hi, i'm aware that if certain requests in core:archive fail, it builds up over time in the archive table. I'm trying to run core:purge-old-archive-data to see if it'll make the table smaller, but it OOMs because the table is too large. Could the script be improved to not OOM? Or could the core:archive command be improved not to fill the archive table like it does if a request fails?

For reference here's the sizes for the tables (between july and august)

-rw-rw---- 1 mysql mysql 2.4K Aug 26 18:39 piwik_archive_blob_2019_07.frm
-rw-rw---- 1 mysql mysql 652M Aug 26 18:40 piwik_archive_blob_2019_07.ibd
-rw-rw---- 1 mysql mysql 2.4K Aug 26 22:16 piwik_archive_blob_2019_08.frm
-rw-rw---- 1 mysql mysql 3.8G Aug 31 12:52 piwik_archive_blob_2019_08.ibd

-rw-rw---- 1 mysql mysql 2.8K Aug 26 18:45 piwik_archive_numeric_2019_07.frm
-rw-rw---- 1 mysql mysql 288M Aug 26 18:45 piwik_archive_numeric_2019_07.ibd
-rw-rw---- 1 mysql mysql 2.8K Aug 26 22:15 piwik_archive_numeric_2019_08.frm
-rw-rw---- 1 mysql mysql 1.4G Aug 31 12:52 piwik_archive_numeric_2019_08.ibd

@tsteur commented on September 1st 2019 Member

What's your memory set to?

@paladox commented on September 1st 2019 Contributor

We have 2gb of ram with most of it being used by redis. The memory we have set is 128M.

@tsteur commented on September 2nd 2019 Member

@paladox I recommend you set it to 256MB then. We would maybe start investigating if it was more than 512MB or so. Anything less than 256MB doesn't worry us all that much. Especially considering this is not executed very often etc. If you're keen on providing a PR that has same functionality but uses less memory we'd be happy to give the PR a review 👍

This Issue was closed on September 2nd 2019
Powered by GitHub Issue Mirror