Historical data for "Yesterday" may not always be processed #18344

Refs DEV-2422 and potentially #18343

When a plugin that uses the rearchive_reports_in_past_last_n_months config setting and the setting is non-zero (Enabled) it seems as though the archiver skips processing the report for "Yesterday"

This was initially reported by a customer.

The easiest way to test this is with a Custom Report. As we can see in the screenshot below, this custom report was created on the 11th of November and according to the screenshot the 10th of November has no report data.

I tested this in my own instance and created a new Custom Report, the date of my server was the 14th of November and as we can see from the attached logs, the report for the 13th of November (Yesterday at the time of creation) was not processed for the Custom Report:
core-archive-custom-reports-yesterday.log

This Custom report has the following creation date:

select created_date from matomo_custom_reports where idcustomreport = 30;
+---------------------+
|created_date|
+---------------------+
|2021-11-14 22:31:21 |

It seems that it’s maybe skipping “Yesterday” because it would be processed during the normal core:archive run. But this might not always happen if the reports were already processed that day - in which case the “Yesterday” reports were maybe already processed and would maybe not be processed again.

Possible that changing Date::yesterday() to Date::today() helps in https://github.com/matomo-org/matomo/blob/4.6.0-rc1/core/Archive/ArchiveInvalidator.php#L460

Potentially the user or you @Starker3 could even try changing this and see if that works?

@tsteur I quickly tested and it looks like it would work just changing that:

INFO [2021-11-18 01:02:11] 2930698    Will invalidate archived reports for today in site ID = 1's timezone (2021-11-18 00:00:00).
INFO [2021-11-18 01:02:11] 2930698    Will invalidate archived reports for yesterday in site ID = 1's timezone (2021-11-17 00:00:00).
INFO [2021-11-18 01:02:11] 2930698  Archived website id 1, period = day, date = 2021-11-18, segment = '', 0 visits found. Time elapsed: 0.518s
INFO [2021-11-18 01:02:11] 2930698  Archived website id 1, period = day, date = 2021-11-17, segment = '', plugin = CustomReports, report = 32, 0 visits found. Time elapsed: 0.518s
INFO [2021-11-18 01:02:11] 2930698  Archived website id 1, period = day, date = 2021-11-16, segment = '', plugin = CustomReports, report = 32, 4 visits found. Time elapsed: 0.518s

Great, thanks for confirming. I would create a PR for this but it also requires to make the tests work again so it's not done in 2 minutes.