The goal of this issue is to modify the Segment Editor and let users view and edit a new segment property to let them decide how far back should we go pre-processing the reports data for each segment.

Problem description

Imagine a very big Piwik instance with maybe hundreds of websites and millions of monthly actions measured. Then imagine one Super User creating a new segment "Enabled for all websites". By default, when creating a custom segment in Piwik, when core:archive next runs, it will pre-process all of the historical data. This is because the core:archive command calls the Piwik API with eg. period=year&date=last7 which pre-processes each daily report, weekly report, monthly report, and yearly report for the last 7 years. In the use case of hundreds of websites and millions monthly actions, if using Piwik for years, such processing could even take several days to complete for a given Piwik instance. This kinda breaks the flow and then Piwik admins have to stop the cron and set custom parameters to work around this, which is very cumbersome.

When this problem occurs

The use case we care about here is the High traffic Piwik use case:

• browser archiving is disabled in settings
• and config INI browser_archiving_disabled_enforce=1 is set

Then we expect that Piwik load is controlled, but still it is not due to the new segments possibly added overnight causing the un-controlled load.

Mitigation

In #7223 we already introduced a new INI setting to decide Piwik-wide the "minimum date" that should be pre-processed for Custom Segments. This is useful, but sometimes not good enough when we want to control such setting per-segment.

Make it available on per-segment basis via UI

The goal of this issue is to add this feature in the UI (and API) on per-segment basis.

For example something like: Process historical report data: [...] where users could choose between

• from beginning of time (default)
• from this segment's creation date YYYY-MM-DD
• from [ 3 ] months before this segment's creation date

Notes

• The (default) value is based off the INI setting process_new_segments_from set to beginning_of_time by default in 2.12.0.
• from [ N ] months before this segment's creation date: we want to provide users a good set of choice. for some segments we want [ 3 ] months before creation date (a bit of historical data), for others we want [ 18 ] months of historical data... eg. we could use the select box from bottom right of reports, and have a month count selector eg. [ 3, 6, 9, 12, 15, 18, 24, 36, 48 ].
• When a segment is created with the default value set, then we should not "copy" this value in the segment. The goal is that when the INI setting process_new_segments_from is changed, it would impact all segments that were created with the previous (default) value.