Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All Websites report doesn't use archiving_custom_ranges to pre-process date ranges #19274

Open
Starker3 opened this issue May 26, 2022 · 4 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo.

Comments

@Starker3
Copy link
Contributor

There is this config setting that allows specific date ranges to be pre-processed for specific date ranges (I.e. specific date ranges, or periods such as last90 etc.):

; By default Matomo will automatically archive all date ranges any user has chosen in their account settings.
; This is limited to the available options last7, previous7, last30 and previous30.
; If you need any other period, or want to ensure one of those is always archived, you can define them here
archiving_custom_ranges[] =

However, this config setting doesn't seem to work to pre-process the All Websites dashboard for the specified Date Ranges.

@Starker3 Starker3 added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label May 26, 2022
@bx80
Copy link
Contributor

bx80 commented May 29, 2022

@Starker3 I'm not sure there is current any archiving of multi-site data, the datatables for the sites appear to be loaded and merged on demand to create the all website view.

To support pre-processing of multi-site data the multi-sites plugin would need to be enhanced with it's own archiving processor to write the merged sites data to an archive.

This could provide a performance benefit in some circumstances but could also increase the load on the archiving processes.

@bx80 bx80 added this to the For Prioritization milestone May 29, 2022
@bx80 bx80 added c: Performance For when we could improve the performance / speed of Matomo. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels May 29, 2022
@Starker3
Copy link
Contributor Author

@bx80 Do you know if the datatables that are loaded when the all websites dashboard is loaded is getting data from the already archived periods?
I don't know if its worth the effort to implement some sort of multi-set pre-processing as long its its able to load the pre-processed reports for the period selected.
(I.e. is it currently treating a date range report the same way it would for a date range that isn't yet processed even if the individual sites already have those periods processed?)

@bx80
Copy link
Contributor

bx80 commented May 31, 2022

@Starker3 Existing numeric archives are loaded individually for each site and then merged into a single datatable. From what I can see, if there are existing archives for the requested date range for all the sites then they should be used.

It's worth noting that previous period data tables are also loaded for the all websites dashboard, so if there are 10 sites then 2 x 10 archives would be loaded and merged into two datatables. I'm not sure what would happen trying to get the previous period of last90, this might cause (slow) archiving of that date range.

It would probably take a bit of time to investigate this in more detail.

@Starker3
Copy link
Contributor Author

Starker3 commented Jun 6, 2022

@bx80 Thanks for the info, so it sounds to me that the pre-processed archives are actually used, but it still takes a bit of processing to get each of them and to combine them into a single report for the "All Websites" dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

No branches or pull requests

2 participants