Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Take archive TTL time into consideration for today when browser archiving is enabled #18241

Merged
merged 1 commit into from Nov 8, 2021

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Oct 29, 2021

Description:

While we looked into the goal range issue this looked potentially like a bug to me.

There is a return false; but at the end of the method also a return false; so I was wondering why the check was there in the first place as it would return false anyway.

Then looked at the check $minDatetimeArchiveProcessedUTC && Date::factory($tsArchived)->isEarlier($minDatetimeArchiveProcessedUTC) and if I read this right, if the archive was created before the minimum date, then it doesn't force archive invalidation but probably it should.

Could maybe someone have a look that this PR looks correct? Could think about adding a test but this might be tricky. I haven't even tried to reproduce the issue due to lack of time. I would think that

  • If browser archiving is enabled
  • And you view an older period
  • And you are viewing today
  • And there is already an archive that is older than the configured [General]time_before_today_archive_considered_outdated setting

Then I would expect that maybe Matomo falsely isn't archiving again meaning effectively Matomo will only archive again if the report is being requested the next day again. But only max once per day maybe.

Review

@tsteur tsteur added the Needs Review PRs that need a code review label Oct 29, 2021
@tsteur tsteur changed the title Take archive TTL time into consideration Take archive TTL time into consideration for today when browser archiving is enabled Oct 29, 2021
@sgiehl
Copy link
Member

sgiehl commented Oct 29, 2021

From a logically point of view that change makes sense to me. maybe @diosmosis can quickly confirm that

@diosmosis
Copy link
Member

@sgiehl yes, makes sense to me too 👍

@tsteur
Copy link
Member Author

tsteur commented Oct 31, 2021

I was actually trying to reproduce that issue but couldn't.

The minArchiveTime was already handled in https://github.com/matomo-org/matomo/blob/4.6.0-b2/core/DataAccess/ArchiveSelector.php#L114-L118

As you can see the first parameter is false for $idArchive in that case. However, this method shouldForceInvalidatedArchive will be only called when $idArchive is not empty in https://github.com/matomo-org/matomo/blob/4.6.0-b2/core/ArchiveProcessor/Loader.php#L128-L132

Meaning when the archive is older than the min time, it will actually never go into that code.

None of the previous return statements in Loader.php return a non-empty idArchive so I'm thinking we could even remove that if statement where I made the change. Or we could just merge as a fallback as it might make sense.

It was added in https://github.com/matomo-org/matomo/pull/17379/files for ranges. Also tried to reproduce it for ranges where it includes today, but failed to reproduce to get it into that method.

Also tested eg with various time_before_range_archive_considered_outdated = 1|900|-1 settings.

I suppose we could just merge it and it'll be safe but not entirely sure why it is there.

@sgiehl
Copy link
Member

sgiehl commented Nov 2, 2021

Guess having it as fallback might not hurt. We could maybe add a comment there, so someone looking at that code might directly notice it might be unneeded.
@tsteur could we maybe add some logging on cloud within that if, to check if it is ever triggered there?

@tsteur
Copy link
Member Author

tsteur commented Nov 2, 2021

Good idea @sgiehl added it. Will check in the evening if anything came up.

@tsteur
Copy link
Member Author

tsteur commented Nov 3, 2021

No log so far, will check again tomorrow

@tsteur
Copy link
Member Author

tsteur commented Nov 3, 2021

@sgiehl ran this for almost 24 hours and it was not triggered once. I'll check again tomorrow

@tsteur
Copy link
Member Author

tsteur commented Nov 4, 2021

@sgiehl last 24 hours it was again not logged. I can check again next Monday. On Cloud this would go in there only for ranges. Possible they aren't being invalidated that often or so.

@tsteur
Copy link
Member Author

tsteur commented Nov 4, 2021

AKA it might be more reproducible that the code is needed on a Matomo where browser archiving is enabled. Which it isn't on Cloud

@tsteur
Copy link
Member Author

tsteur commented Nov 7, 2021

Still nothing in the logs @sgiehl but then we also don't have browser archiving enabled.

we could either remove the if check (as it currently does nothing anyway) or merge the PR.

as it the check was added I'm assuming there was a reason for it and we'd need to change to return true but I don't really have any preference there.

@sgiehl
Copy link
Member

sgiehl commented Nov 8, 2021

I'll simply merge it.

@sgiehl sgiehl merged commit 27d714e into 4.x-dev Nov 8, 2021
@sgiehl sgiehl deleted the min_archive_time_consideration branch November 8, 2021 08:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants