@tsteur
Description:
See this profile run where we fetch data over 9-10 months:
It issues 281 SQL queries in this part of the code. Then set a breakpoint as this wasn't expected and it's actually fetching a query per period (eg per day or per week) when it only needs to fetch one query per month.
See screenshot below. Before it would execute 10 queries (one per day period), vs now it would only fetch 2 queries and in the second query fetch 9 of them.
It will now basically aggregate all entries from same archive data into one query. Looking at the code I believe it was actually meant to work like that but never really was.
Review
- [ ] Functional review done
- [ ] Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
- [ ] Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
- [ ] Security review done
- [ ] Wording review done
- [ ] Code review done
- [ ] Tests were added if useful/possible
- [ ] Reviewed for breaking changes
- [ ] Developer changelog updated if needed
- [ ] Documentation added if needed
- [ ] Existing documentation updated if needed
@sgiehl
@tsteur The code changes are looking fine.
But are you able to explain why the test results for that specific tests changed?
@sgiehl yes, I am.
First of all, there are 2 same entries in the log file for different domain. So /blog/category/meta/index is available for both domains.
Then previously see below we would have fetched one archive after another for each day. First idArchive 13, then idArchive 5, then idArchive 8 but now we group this into one query to fetch them 13,5,8 at in one query.
idArchive being an in index etc it returns the archive rows sorted first 5 then 8 then 13. Vs before the $rows would have been 13,5,8.
That means the original URL from a different domain gets preference. There isn't generally one more right than the other though and things can be always completely random depending on what day what was tracked first. We can't even order by idArchive or ts_archived as things might be archived completely out of order. Generally, both are "correct" though so this shouldn't be needed anyway.
Ok. That makes sense :+1: