Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rendering performance improvement for reports with many rows #18394

Closed
Starker3 opened this issue Nov 28, 2021 · 3 comments · Fixed by #18405
Closed

Rendering performance improvement for reports with many rows #18394

Starker3 opened this issue Nov 28, 2021 · 3 comments · Fixed by #18405
Assignees
@bx80
Labels
c: Performance For when we could improve the performance / speed of Matomo. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone
4.7.0

Comments

@Starker3
Copy link
Contributor

Refs L3-181

When loading User ID reports with several thousand rows (For example 3k-10k rows) the report generation in the browser causes very high system resource usage on the device loading the report.

For example an matomo server that has a User ID row limit of 10,000 and selecting the row limit in the UI to "All" takes anywhere from 3-6 minutes (or more) to generate the report in the browser. During this time, that browser tab uses for example 3gb+ of RAM and presents the popup "Browser tab is unresponsive"

If the device being used doesn't have enough spare RAM or just isn't powerful enough to run the javascript to generate the report, that report can basically get stuck for that user and would force them to reset their visualisations to fix it (https://matomo.org/faq/troubleshooting/what-do-i-do-when-the-browser-tab-becomes-unresponsive/).
@Starker3 Starker3 added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Nov 28, 2021
@Starker3 Starker3 changed the title User ID reports slow to load and cause very high browser resource User ID reports slow to load and cause very high browser resource usage Nov 28, 2021
@tsteur tsteur added c: Performance For when we could improve the performance / speed of Matomo. Regression Indicates a feature used to work in a certain way but it no longer does even though it should. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels Nov 28, 2021
@tsteur tsteur added this to the 4.7.0 milestone Nov 28, 2021
@tsteur
Copy link
Member

tsteur commented Nov 28, 2021

I think we're seeing this also in flattened page url reports etc.

We'd want to reproduce this, and then possibly check in what version this regressed if it did regress at all. Or if it's done quicker we could fix it directly.

Below a profile.
image

Back in the days we had to do column highlihting etc using JS. Not sure if that's still needed now that we don't support certain browsers anymore.

@bx80
Copy link
Contributor

bx80 commented Nov 30, 2021

@tsteur I've reproduced this on a local dataset with ~6,800 user ids.

It takes 151 seconds to fully render the user id report using almost 4gb of memory for the browser tab (firefox).
Browser profiling shows that 98 seconds (~65%) of the total time is spent on the dataTable.js::handleColumnHighlighting() method.

image

To confirm this I added a return statement at the top of the handleColumnHighlighting() method which reduced the total report rendering time to 69 seconds using less than 2gb tab memory.

image

handleColumnHighlighting() is setting column width (via iterating over rows!) and handling row highlighting with mouseenter and mouseleave events. Disabling the method doesn't appear to make any difference to column highlighting or column widths they are being handled by CSS.

I'll create a PR to remove this method entirely and will do some browser testing with the minimum supported browsers for Matomo v4.6.0

@bx80 bx80 mentioned this issue Nov 30, 2021
Merged
11 tasks
@tsteur
Copy link
Member

tsteur commented Nov 30, 2021

👍 be good to potentially also check if we can make the ellipsis work with pure CSS (assuming they take also like 30% of the CPU for you) these days but I believe this method by now does a lot more maybe not just to make ellipsis work.

@bx80 bx80 self-assigned this Dec 1, 2021
@bx80 bx80 linked a pull request Jan 4, 2022 that will close this issue
Rendering performance improvement for reports with many rows #18405
Merged
11 tasks
@bx80 bx80 closed this as completed Jan 4, 2022
@justinvelluppillai justinvelluppillai changed the title User ID reports slow to load and cause very high browser resource usage Rendering performance improvement for reports with many rows Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bx80 bx80

Labels
c: Performance For when we could improve the performance / speed of Matomo. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Milestone
4.7.0
Development

Successfully merging a pull request may close this issue.

Rendering performance improvement for reports with many rows matomo-org/matomo
3 participants
@tsteur @Starker3 @bx80