Fix today archive is not being archived #15996
Conversation
Noticed an error that today wasn't being archived. Eg I had set config to: ```ini enable_general_settings_admin=0 time_before_today_archive_considered_outdated = 1 ``` I would have expected that the archive is being invalidated every time I execute the console archive command but this wasn't happening. Then noticed `$minDatetimeArchiveProcessedUTC` was already a date and therefore somehow converted again resulting in a different time.
tsteur
added
not-in-changelog
|
@diosmosis I think there's also still another bug. Maybe around https://github.com/matomo-org/matomo/blob/todayarchive/core/CronArchive.php#L876-L880 When I set the TTL for archives to |
core/DataAccess/ArchiveSelector.php
Outdated
| @@ -92,10 +92,14 @@ public static function getArchiveIdAndVisits(ArchiveProcessor\Parameters $params | |||
| return [false, $visits, $visitsConverted, true, $tsArchived]; | |||
| } | |||
|
|
|||
| if (!empty($minDatetimeArchiveProcessedUTC) && (!is_object($minDatetimeArchiveProcessedUTC) || !$minDatetimeArchiveProcessedUTC instanceof Date)) { | |||
There was a problem hiding this comment.
If the var isn't an object then it will never be a Date, right?
There was a problem hiding this comment.
Could remove it. Guess in case it was some different class but that wouldn't work anyway
I'm not sure what you mean by "ignore parameter", that value is whether we found any archive at all for the period/site/segment/plugin. It's used in Loader.php to avoid invalidating if there's no archive at all. |
This would mean that the archive invalidation isn't invalidating week/month/year as well. Are you able to check if after that code is run, there are entries for those in archive_invalidations? And that the archives are set to DONE_INVALIDATED? If there are, then there's an issue in |
Ignore this. I was reading it wrong. There is no entry afterwards in the invalidations table and this is what I got as output: |
|
|
Be good if you could have a look. I recommend above mentioned ini settings as then we'd expect today, this week, this month, this year to be always invalidated |
|
@tsteur not able to reproduce it. If I exit after |
|
Don't know why (eg whether it is maybe a timezone issue) but now seeing these invalidations when exiting after the invalidations:
However, I do then a full archive using Then I trigger Can you maybe try to reproduce it that way? |
|
I see, I'll try w/ a 1s ttl. |
|
@tsteur can't reproduce still. Can you run the command w/ |
|
@diosmosis it thinks there is no new visit even though there is... will do a quick debug |
|
Actually it works now also for me. And it correctly detected it had visits for idSite=1 it was just not mentioned because the output starts with idSite1 and I thought all messages below belonged to idSite 1 but they were for other idSites (might need to add this to the debug messages) So seems to work 👍 |
|
fyi i think what happened was that something removed |
|
👍 the output could probably be improved. I'm currently working on this in another PR. |
|
👍 I guess we can merge this then? |
@diosmosis Noticed an error that today wasn't being archived while testing again #15991 . Eg I had set config to:
Timezone of the site:
America/Los_AngelesI would have expected that the archive is being invalidated every time I execute the console archive command but this wasn't happening. Then noticed
$minDatetimeArchiveProcessedUTCwas already a date and therefore somehow converted again resulting in a different time.See screenshot where the archive should have been ignored because
ts_archivedis older but it wasn't ignored.See what happened to the date after passing it again to
date::factory:Looks actually a bit like a bug to me that
Date::factorywould set time to00:00:00.Another potential bug maybe @diosmosis ... the last line in that method returns
return [$idArchive, $visits, $visitsConverted, true, $tsArchived];. Should the ignore parameter really be set totrue?