Hi, i'm aware that if certain requests in core:archive fail, it builds up over time in the archive table. I'm trying to run core:purge-old-archive-data to see if it'll make the table smaller, but it OOMs because the table is too large. Could the script be improved to not OOM? Or could the core:archive command be improved not to fill the archive table like it does if a request fails?
For reference here's the sizes for the tables (between july and august)
-rw-rw---- 1 mysql mysql 2.4K Aug 26 18:39 piwik_archive_blob_2019_07.frm
-rw-rw---- 1 mysql mysql 652M Aug 26 18:40 piwik_archive_blob_2019_07.ibd
-rw-rw---- 1 mysql mysql 2.4K Aug 26 22:16 piwik_archive_blob_2019_08.frm
-rw-rw---- 1 mysql mysql 3.8G Aug 31 12:52 piwik_archive_blob_2019_08.ibd
-rw-rw---- 1 mysql mysql 2.8K Aug 26 18:45 piwik_archive_numeric_2019_07.frm
-rw-rw---- 1 mysql mysql 288M Aug 26 18:45 piwik_archive_numeric_2019_07.ibd
-rw-rw---- 1 mysql mysql 2.8K Aug 26 22:15 piwik_archive_numeric_2019_08.frm
-rw-rw---- 1 mysql mysql 1.4G Aug 31 12:52 piwik_archive_numeric_2019_08.ibd
What's your memory set to?
We have 2gb of ram with most of it being used by redis. The memory we have set is 128M.
@paladox I recommend you set it to 256MB then. We would maybe start investigating if it was more than 512MB or so. Anything less than 256MB doesn't worry us all that much. Especially considering this is not executed very often etc. If you're keen on providing a PR that has same functionality but uses less memory we'd be happy to give the PR a review 👍