Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improves memory usage of CSV exporter #19315

Merged
merged 1 commit into from Jun 7, 2022
Merged

Conversation

sgiehl
Copy link
Member

@sgiehl sgiehl commented Jun 3, 2022

Description:

When exporting e.g. Live.getLastVisitsDetails with a result that contains a lot visits and some visits that have hundreds of actions, this can currently cause an out of memory problem, as iterating over all the data and extending the result array with thousands of empty keys, bloats up the array too much.

Did some local testing having a visit with more than 300 actions and a result set of only around 100 visits.

Before the changes of this PR the API export had a memory peak of around 73.762.528 Bytes
With the changes the peak went down to only 46.386.536 Bytes.

fixes #19158

Review

@sgiehl sgiehl added the Needs Review PRs that need a code review label Jun 3, 2022
@sgiehl sgiehl added this to the 4.11.0 milestone Jun 3, 2022
@sgiehl sgiehl marked this pull request as ready for review June 3, 2022 13:12
@justinvelluppillai justinvelluppillai merged commit 42870c6 into 4.x-dev Jun 7, 2022
@justinvelluppillai justinvelluppillai deleted the improvecsvexport branch June 7, 2022 02:27
@justinvelluppillai justinvelluppillai added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. and removed Needs Review PRs that need a code review labels Jun 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve memory management in DataTable/CSV
2 participants