@tsteur opened this Issue on August 25th 2019 Member

We can already configure time_before_today_archive_considered_outdated = XYZ to ensure we regenerate an archive only every XYZ seconds.

Sometimes, when you have segments, you may not want to process them as often though as these are maybe viewed less often or because they are expensive to archive. So you maybe want to archive today every 30 minutes, but segments only every 2 hours.

It be good to have a setting time_before_today_segments_archive_considered_outdated = XYZ which only applies to archives with segments.

I'm seeing in the code Piwik\ArchiveProcessor\Rules::getTodayArchiveTimeToLive() is used in quite a few places and all the calling methods would need to be tweaked to also parse a segment instance (or segment string or something) as a parameter so we can read the correct config value. Seems also archive purging might depend on this where we maybe want to use for simplicity (min|max(todayArchiveTimeOutdated, todaySegmentArchiveTimeOutdated)) if it makes it any simpler.

We also need to make sure that when a new segment is being added, it will be archived right away the next time core:archive runs. So we need to make sure time_before_today_archive_considered_outdated does not block the archiving of a new segment.

Powered by GitHub Issue Mirror