Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Faster segmented suggested values when browser archiving is disabled #15732

Closed
tsteur opened this issue Mar 25, 2020 · 0 comments
Closed

Faster segmented suggested values when browser archiving is disabled #15732

tsteur opened this issue Mar 25, 2020 · 0 comments
Assignees
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Milestone

Comments

@tsteur
Copy link
Member

tsteur commented Mar 25, 2020

We can also do this as part of 3.13.6 if needed.

We're getting the feedback around slow segment suggested values a lot and it can potentially even take down Matomo when it needs to look at a lot of data over 60 days. I'm meaning the API.getSuggestedValuesForSegment method.

We would like the ability that for some segments to optionally define an API method (eg Events.getAction).

  • should browser archiving be disabled, then we would get the suggested values by looking at the most common labels instead of issuing the live query
  • should browser archiving be enabled, then this can be slow to start archiving so we would prefer the live visitor log query instead to avoid slow archiving

We'll need to make sure to have tests for both ways for this API method. Initially we would probably not set this for visitor segments but action segments were possible (page urls, page titles, events, ...). Could set it for some visitor segments though as well.

Ideally, we would not need to do range archives eg period=range&date=last2months. Looking at one full month be fine. However, we can't really look at previous month since then recent values would be missing and we can't look at the current month because we might be only a few days into the month.

  • I suggest looking at this year's data be fine when browser archiving is disabled.
  • When the current month is January, then we would maybe need to look at last 2 months (this and previous month) and accept there will be a range archive. We wouldn't want to use last 2 years as it would be slower to aggregate since year archives usually have a lot more data.

From a technical point of view:
The idea is, that a Dimension class would be able to define an API method eg protected $segmentSuggestedValuesApi = 'Actions.getPageUrls'. This would be automatically passed on to the segment class and in the API.getSuggestedValuesForSegment we can then check if browser archiving is disabled and if there is an API defined, then we issue the API request instead of a live query.

We would also specify all needed parameters like the filter_limit=X, filter_offset=0, segment = '' in Request::processRequest.

As a side effect, should range period be disabled, this would make the suggested values still work for the months from Feb-Dec. Jan wouldn't work. We need to make sure to have a check for this like

        if (!Period\Factory::isPeriodEnabledForAPI('range')) {
            return array();
        }

to prevent any exception.

@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label Mar 25, 2020
@tsteur tsteur added this to the 3.13.5 milestone Mar 25, 2020
@tsteur tsteur self-assigned this Apr 7, 2020
@sgiehl sgiehl closed this as completed Apr 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

No branches or pull requests

2 participants