Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transitions report very slow or fails to load without extra INDEX #14762

Open
mattab opened this issue Aug 13, 2019 · 3 comments
Open

Transitions report very slow or fails to load without extra INDEX #14762

mattab opened this issue Aug 13, 2019 · 3 comments
Labels
c: Performance For when we could improve the performance / speed of Matomo. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.

Comments

@mattab
Copy link
Member

mattab commented Aug 13, 2019

This is a long known issue: Transition report can be very slow to run. Often even times out as it doesn't complete in 30 or 60 seconds. As far as I know, transitions loading slowly is the only part of Matomo that is consistently and predictably slow (on any medium or large site. does not have to be a huge website to experience this slowdown or malfunction).

We document the workaround in FAQ: https://matomo.org/faq/how-to/faq_161/

Now that we improved so much of Matomo and everything else should load really fast, it feels unacceptable whenever Transitions takes 30 or 60 or 120 seconds to load. How many hundreds/thousands of people get frustrated when using transitions? Knowing there is a solution, it feels un-necessary. The solution involves adding several INDEX which will make tracking slightly slower. But I reckon the extra INDEXes are worth the benefits of having an always-fast Matomo UI.

--> My suggestion is to add the Transitions index into core for everyone while upgrading to Matomo 4.0.0:

ALTER TABLE  `matomo_log_link_visit_action` ADD INDEX `transitions_url` (  `idaction_url` ,  `idsite` ,  `server_time` );
ALTER TABLE  `matomo_log_link_visit_action` ADD INDEX `transitions_url_ref` (  `idaction_url_ref` ,  `idsite` ,  `server_time` );
ALTER TABLE  `matomo_log_visit` ADD INDEX `transitions` (  `visit_entry_idaction_url` ,  `idsite` ,  `visit_last_action_time` );

(We would also want to create the index for all cloud customers, as even on our own instance (not large traffic) Transitions takes 60+ seconds to load for a yearly report.)

@mattab mattab added Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. c: Performance For when we could improve the performance / speed of Matomo. labels Aug 13, 2019
@mattab mattab added this to the 4.0.0 milestone Aug 13, 2019
@tsteur
Copy link
Member

tsteur commented Aug 13, 2019

I would definitely not want to see these indexes there as it makes things so much slower in the critical tracker part. There must be other ways to solve this. Be good to investigate how this could be made faster without needing new index.

Also in general in case we did as part of the hot/cold table issue #14119
have one log table with recent values where we track into, then we wouldn't need the indexes on that table as it would be likely fast there and would only need the index on the table that holds all other entries. It would at least not slow tracking down this way etc.

Nonetheless there might be other ways to improve this. Eg can we find an efficient way to archive this for say the top 1K page urls etc

@MatomoForumNotifications

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/transitions-execution-time-oops/45054/3

@webberian
Copy link

Also see discussion in #14172.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests

5 participants