Optimized how archive data is fetched from database #7023
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Now we try to make only 1 query per archive table (before there could be multiple queries per table). In weekly archiving with very few data the number of SQL queries went from 2500 to 1000, saved 6% of execution time, 5% of memory usage.
mnapoli
added
c: Performance
|
Looks like there's a bug in Archive.php somewhere, from the code, the result of Archive::getArchiveIds() is passed to ArchiveSelector::getArchiveData(), and Archive::getArchiveIds() returns Archive::getIdArchivesByMonth(), so the intent is to return an array whose keys have a 1-1 correspondence w/ archive tables. So the code should already do only one query per table, but something is causing it not to. |
|
👍 thanks for the insight, I'll try to have a look at it |
mnapoli
added
the
Pull Request WIP
|
Moving into 2.12 but @mnapoli feel free to take a look prior if you can. Also we can look together if it helps! |
|
Ah yeah I had a look at the code last month but couldn't find anything to explain why there are duplicate queries. Maybe someone else wan't to have a look. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
References #6846
Now we try to make only 1 query per archive table (before there could be multiple queries per table).
In weekly archiving with very few data the number of SQL queries went from 2500 to 1000, saved 6% of execution time, 5% of memory usage. Real results could be much less significant (or more) I have no idea…
Also, please review the change because:
I juste tried to optimize that bit from what I could understand.
We could also consider issuing only 1 MySQL request by selecting from all tables at once using
UNION, by I don't know if it could explode because of the amount of data…