@Dave-Oz As long as you do not delete any of the log data all archives can be rebuilt at any time. Nevertheless purge-old-archive-data should only remove archives that aren't needed anymore.

@Dave-Oz As long as you do not delete any of the log data all archives can be rebuilt at any time. Nevertheless purge-old-archive-data should only remove archives that aren't needed anymore.

@sgiehl We have Regularly delete old raw data from the database option on and it's set to 60 days in the settings.

Also, there is the "Schedule old data deletion" is set to every week in the settings.

Does that mean matomo was supposed to that for January? However, it didn't do it for some reason (a bug in an older version)?
So, if I do it manually, it should be fine?

Also, we have cron task that runs every couple minutes with the command /matomo/console core:archive

I'm just trying to understand what is the difference in that table before and after running purge-old-archive-data january?
What did it get rid of and the size went from 4.4gb to 16mb?

Which table stores the log data archives?
What is stored in piwik_archive_blob_2020_01? Do you mean that piwik_archive_blob_2020_01 can be created successfully anytime based on the data stored in some other table already in the database without a loss of information/metric data?

If you archive data every few minutes then over time there will be a lot of outdated reports in the DB which some get deleted daily, some weekly, some monthly. If the size is an issue then I recommend running the archive command for example only every hour.

The archive tables store the reports. You can find more info in https://developer.matomo.org/guides/how-piwik-works#data-model-processing-and-storage

Does that mean matomo was supposed to that for January? However, it didn't do it for some reason (a bug in an older version)?

Yes there was a bug in an older version which we fixed in an update and the update should have triggered the cleanup to run eventually (but might take a while until the task to clean up is executed the next time).

@tsteur Our only issue is the size for January 2020. All the other months are below 30-40mb each. Except January 2020. That one is over 4gb.
After I ran the command below. This is the output.
It spent a lot of time purging the invalidated archives.

# ./console core:purge-old-archive-data january
Purging outdated archives for 2020_01...Done. [Time elapsed: 0.110s]
Purging invalidated archives for 2020_01...Done. [Time elapsed: 3254.697s]
Purging custom range archives for 2020_01...Done. [Time elapsed: 0.618s]
Optimizing archive tables...
Optimizing table piwik_archive_numeric_2020_01...Done. [Time elapsed: 12.164s]
Optimizing table piwik_archive_blob_2020_01...Done. [Time elapsed: 243.220s]

My only worry is that does this break annual reports since they are stored in January 2020?

@tsteur Also, which task to clean up is executed? Can I do that manually? Is it equivalent to ./console core:purge-old-archive-data ?

./console core:purge-old-archive-data january

should do 👍 it will only delete unneeded data. You could also try

./console core:purge-old-archive-data --include-year-archives today

Then it does the current month and the 2020_01 table.

@tsteur Hi,
I did that. It's been 3-4 days. Everything has been fine. However, 2020_01 started to grow again. It was 16mb when I first executed "./console core:purge-old-archive-data January" and now it reached to 200+ MB. I think annual reports are causing this. Is there anything else that I can use to make it stop growing into gigabytes again?

You could execute this task as a cron or run the archiver less often @Dave-Oz

Running it every few minutes is quite often.

--

@unkn0wn-developer I see you removed your comment maybe it's all good?

@unkn0wn-developer I see you removed your comment maybe it's all good?

It's all good for now. I'm monitoring the usage every week or so. I wasn't using the "ls" command in MB or GB mode. I thought I saw that the backup size was 5gb but it was 500mb. Counted the digits wrong in bytes... It was a Friday evening at work so my brain was pretty much fried at the end of the day.😅 Thanks for checking up on me though.