The goal of this issue is to help users who have poor experience of Piwik by detecting when it happens and displaying a help notification to guide them in right direction.
Problem: Piwik loads stats slowly for many users when Browser trigger archiving is enabled in settings. It affects mostly
All Websitesdashboard, detect when stats take more than a few seconds to load and display inline notification suggesting to solve the issue.
Last 30 daysthen we should pre-process it anyway, see #6672
How often should the notification be displayed? if the user closes the notification, maybe it could be displayed again in one week. The notification should be displayed to all users but messaging should change for Super User to recommend them to change the settings or setup cron (for normal users it would recommend to contact the Piwik Administrator to let them know of the issue and suggest them to tweak Piwik).
Who is affected by issue?
Almost all users by default will experience this issue sooner rather than later. By default the automated cron is not setup and user has browser archiving enabled in piwik settings. by default we load Today, and All websites can also load the Last 30 days, so the user will request a lot of fresh data most of the time!
Maybe you have other ideas?
Two small notes:
I'm not sure what is default date selection but for sure having "yesterday" in general would be better default than "today".
Today we got again some user feedback of frustration at the slowness of the
All Websites dashboard, here it is:
This All Sites STILL does not work. Nine times no, once yes. PHP has got 512 MB RAM, no errors are shown … it’s kinda joke. This is from the beginning crippled. The old Piwik worked, since this new – broken by design!
In some way it's true that it's broken by design because we do not tell users if they don't look in the settings or the docs.
Because we all know the issue can be solved by setting up the cron, I've added a
Major tag to this issue and updated the description to describe a specific change we could make:
Detect when stats take more than a few seconds to load and display inline notification suggesting to solve the issue.
from forum feedback: http://forum.piwik.org/read.php?2,125969,page=1#msg-126260
I'd regard myself as a representative user. I can follow the install instructions, but they took me a lot longer than 5 minutes. I am not a Linux wiz. I may be mistaken, but I am pretty sure they don't conclude with set-up the cron for auto-archiving otherwise I would have tried or paid someone to help me. My suggestion is that you do add this setup to the instructions. I think it will provide a better experience to your users, and as far as I can tell, there is no downside.
Maybe we could also double check the wording in installer. Right now it says "setup the cron if you have a high traffic website" or so, instead we could make it clear that it's important for fast Piwik experience to setup the cron for most websites (except for very low traffic Piwik.)
FYI: not sure how this could be implemented... Could we hook into our ajax global queue, and/or the AngularJS http queue, to detect if any pending request is taking more than N ~ 15 seconds?
Here is something maybe relevant for this problem: for our UI tests we had to check whether all HTTP requests are finished. When all HTTP are received by the browser then only we can take a screenshot as we know everything is loaded. This wasn't working well in PhantomJS and this is why we had to use our global ajax queue and angular HTTP queue. See the magic code in the UI test helper code: https://github.com/piwik/piwik/blob/2.16.0-rc1/tests/lib/screenshot-testing/support/page-renderer.js#L503-L520
@tsteur considering slow Matomo is the top complaint we hear from on-premise users, would be great to schedule this in Matomo 4.x, as it can usually be easily fixed by disabling browser archiving & setup cron
It be fine as long as we do this easy and simple as a diagnostic check for example. Eg this check runs when Piwik is installed, and we'd always recommend disabling browser archiving (since it always makes it faster and should always be used).
For segment archiving we could also add another warning if enabled and explain the trade off (eg longer wait times vs on demand data and not needing to preconfigure segments).
checking how long a UI call takes etc won't be that helpful.
There is also already a check for this in the "Tour" plugin, and we will be sending out emails mentioning this (we are working on this as part of a different issue).
Later when we go into onboarding improvements we'll see what else we can do.
This feature we can generally easily add any time though so it's not high prio for Matomo 4 and we need to make sure to spend as little time on it as possible.
See e.g. https://forum.matomo.org/t/slow-dashboard-for-sharepoint-2019/37071?u=lukas for an example
It be fine as long as we do this easy and simple as a diagnostic check for example.
Sounds good to have a diagnostics
checking how long a UI call takes etc won't be that helpful.
fyi: additionally to the system check that people may not see (unless they are super users or so, and visit the Admin page). After all if someone has only 10 or 100 visits a day, and don't use segments eg., it's probably mostly loading fast enough for them and wouldn't need to always see a notice at the top. So the idea be to show a message only when it's clearly too slow and they wouldn't know why (maybe even all users would see it like "Reports seem to load slowly. Consider notifying a Super User so they can setup auto-archiving of reports."). But yes this could be done in a later step (and the diagnostic check above added in the meantime.)
If you want any slow detection, then I'll move it out into a different milestone. Also slow is relative and depends on many factors. Generally, we should always recommend cron archiving and most super users would see it eventually. If people have only few visits, they are likely on some slow hoster etc so we should always recommend to get the best Matomo experience as otherwise people will say things are slow to others. These days normal expectations are that things load in 200ms.
Also keep in mind that we will be sending out emails about this etc as part of onboarding etc and will further work on it then.