@mattab opened this Issue on August 25th 2017 Member

In Piwik reports are processed based using the raw data. Once an archive has been processed for a website and a specific date period, and stored in the database (during the core:archive process), it can only be invalidated (via HTTP API or CLI or InvalidateReports plugin) which will force all plugins to re-process all their respective reports in the next archiving process. This is problematic because it can take a lot of time to reprocess previous reports for 3 or 12 months.

-> Plugins should be able to easily create their own reports for some of the historical data. this is useful for all plugins which don't create new raw data but re-uses the existing raw data. For example, Funnels plugin would like to process Funnels data in the past 6 months.

To make it possible for any plugin to append their reports to the existing archives, we need to make some changes to the archiving process.


  • Piwik core to allow an existing archive to be modified later-on so new plugin's reports can be part of and appended in the existing old archive
  • Piwik core to allow each plugin to force re-processing of past data and the ability to specify how far back the data should be re-processed. (working for our two use cases: when using the core:archive console command in a cron, or when using broworser triggered archiving and looking at historical data.)

(not related but a bit similar to https://github.com/piwik/piwik/issues/7573 )

@mattab commented on November 29th 2018 Member

When I create Custom reports, and want historic data, I need to invalidate all historical data, and re-process them. It can takes a very long time and is not practical. Recently we had a problem around too much data invalidated (reported in https://github.com/innocraft/plugin-InvalidateReports/issues/8).

So it will be great to have the ability to "update" existing archives and process the data for specific plugins (eg. custom reports) without having to re-process everything else.

Powered by GitHub Issue Mirror