Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not pre-process custom segments when pre-processing date ranges to load by default #7611

Closed
mgazdzik opened this issue Apr 1, 2015 · 7 comments
Assignees
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone

Comments

@mgazdzik
Copy link
Contributor

mgazdzik commented Apr 1, 2015

In some cases processing every custom range which user selects to display after login may cause huge performance issues. For example in case:

  • 2k websites
  • approx 4k users
  • 500+ segments (for single sites, but still)

When we want to process multiple ranges archiving cannot complete in less than 24 hours (segments are also processed for ranges).
Therefore I propose to introduce config param to prevent this behaviour.

Removing range as available periods will cause UI to break after login attempt, also we don't want to disable segments entirely, so it's not an option.

Please let me know what do you think?

@mgazdzik mgazdzik added the Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. label Apr 1, 2015
@gaumondp
Copy link

gaumondp commented Apr 1, 2015

Did you tried this plugin ?

http://plugins.piwik.org/RerUserDates

"This Piwik Plugin lets you hide custom date range selection from calendar for regular users, also lets you avoid users setting ranges as default value in their profile."

@mattab
Copy link
Member

mattab commented Apr 1, 2015

Removing range as available periods will cause UI to break after login attempt,

covered in #7615

@mattab
Copy link
Member

mattab commented Apr 1, 2015

When we want to process multiple ranges archiving cannot complete in less than 24 hours (segments are also processed for ranges).

Maybe a good trade-off solution would be to not pre-process any segments for those date ranges set as "Default date to load" ?

@mattab mattab added this to the Piwik 2.13.0 milestone Apr 1, 2015
@mnapoli
Copy link
Contributor

mnapoli commented Apr 3, 2015

If #7615 is fixed does that mean this ticket is fixed (because then you can disable ranges)? Or is there something left to do?

@mgazdzik
Copy link
Contributor Author

mgazdzik commented Apr 3, 2015

@mattab - yes, it could be also a reasonable thing to do. I can't tell at this moment how much impact will processing non segmented ranges cause for big instances.
@mnapoli - this ticket is merely about overhead added by processing custom ranges by cron archiving process and processing time increase caused by that

@mattab
Copy link
Member

mattab commented Apr 7, 2015

IMHO it is not a problem to pre-process custom date ranges even with 5000 users or more. At most there will be 6-7 different possibilities of date ranges so it will only add 6-7 requests per website, which is not big.

however there is a large performance overhead with the fact that all custom segments are pre-processed for those custom date ranges (which could often add extra (7 ranges * 200 segments) requests for each website!). We will fix this by not pre-process segments for date ranges.

@mattab mattab changed the title Config option to disable custom ranges processing Do not pre-process custom segments when pre-processing date ranges to load by default Apr 7, 2015
@mattab mattab self-assigned this Apr 7, 2015
@mattab
Copy link
Member

mattab commented Apr 7, 2015

PR: #7635

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

No branches or pull requests

4 participants