Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add possibility to specify a different today archive time for segments #14810

Open
tsteur opened this issue Aug 25, 2019 · 0 comments
Open

Add possibility to specify a different today archive time for segments #14810

tsteur opened this issue Aug 25, 2019 · 0 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo.

Comments

@tsteur
Copy link
Member

tsteur commented Aug 25, 2019

We can already configure time_before_today_archive_considered_outdated = XYZ to ensure we regenerate an archive only every XYZ seconds.

Sometimes, when you have segments, you may not want to process them as often though as these are maybe viewed less often or because they are expensive to archive. So you maybe want to archive today every 30 minutes, but segments only every 2 hours.

It be good to have a setting time_before_today_segments_archive_considered_outdated = XYZ which only applies to archives with segments.

I'm seeing in the code Piwik\ArchiveProcessor\Rules::getTodayArchiveTimeToLive() is used in quite a few places and all the calling methods would need to be tweaked to also parse a segment instance (or segment string or something) as a parameter so we can read the correct config value. Seems also archive purging might depend on this where we maybe want to use for simplicity (min|max(todayArchiveTimeOutdated, todaySegmentArchiveTimeOutdated)) if it makes it any simpler.

We also need to make sure that when a new segment is being added, it will be archived right away the next time core:archive runs. So we need to make sure time_before_today_archive_considered_outdated does not block the archiving of a new segment.

@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label Aug 25, 2019
@mattab mattab added this to the Backlog (Help wanted) milestone Jan 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

No branches or pull requests

2 participants