Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor segment re-archiving in past behavior to be on demand #17005

Merged
merged 21 commits into from Feb 4, 2021

Conversation

diosmosis
Copy link
Member

Description:

Currently we check if a segment has been created or updated since the last time we invalidated in core:archive, and if so invalidate in the past. This is more complicated and error prone, it is far easier to just invalidate in the past in the SegmentEditor API.

Refs #16955

Review

  • Functional review done
  • Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
  • Security review done see checklist
  • Code review done
  • Tests were added if useful/possible
  • Reviewed for breaking changes
  • Developer changelog updated if needed
  • Documentation added if needed
  • Existing documentation updated if needed

@diosmosis diosmosis added the Needs Review PRs that need a code review label Dec 23, 2020
@diosmosis diosmosis added this to the 4.2.0 milestone Dec 23, 2020
@sgiehl
Copy link
Member

sgiehl commented Jan 5, 2021

Wondering if updating that behavior could lead to egde cases on update. E.g. if someone has a cron running only once nightly. After the cron ran a segment is changed and afterwards Matomo is update to that new behavior. The one changed segment won't get invalidated anymore, right? Would we need to do that once in an update maybe? 🤔

@diosmosis
Copy link
Member Author

@sgiehl that makes sense, I'll add an update to check as well.

@diosmosis
Copy link
Member Author

@sgiehl added the update

Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments. Also it seems the branch has unresolved conflicts and some tests are failing...

core/CronArchive/SegmentArchiving.php Show resolved Hide resolved
plugins/SegmentEditor/API.php Show resolved Hide resolved
@diosmosis
Copy link
Member Author

@sgiehl test should be fixed w/ the next build. the last change increases the amount of invalidated dates for segments, which I think is a bug fix. Before we were checking if the segment was created/edited after the earliest date for rearchiving, but it doesn't make sense to do that when invalidating for any report and not invalidating because of a segment.

@sgiehl
Copy link
Member

sgiehl commented Jan 28, 2021

@diosmosis seems there are still some test failures. Also one of my comments is still open. Besides that it would imho be good to merge. Not sure if maybe @tsteur would like to have another look on that.

@tsteur
Copy link
Member

tsteur commented Jan 28, 2021

All good from my side. 👍

@diosmosis diosmosis force-pushed the segment-archiving-in-past-refactor branch from 0e4dddc to 24ff3f4 Compare February 1, 2021 03:25
@diosmosis
Copy link
Member Author

@sgiehl fyi, fixed the tests and in doing so found another bug in the ArchiveInvalidator class. Before we were removing queued invalidations from tracking when invalidating any non-range period. But this meant that if we invalidated just a week, the day wouldn't get re-archived, even if there was a new visit.

Will wait to merge in case you want to do a new review of the change.

core/Archive/ArchiveInvalidator.php Outdated Show resolved Hide resolved
@diosmosis
Copy link
Member Author

@sgiehl updated

@diosmosis diosmosis merged commit 490dc3a into 4.x-dev Feb 4, 2021
@diosmosis diosmosis deleted the segment-archiving-in-past-refactor branch February 4, 2021 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants