Make sure to apply the limit filter on sites and groups #7877
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #7854
Apart from this it also fixes the following issues:
Roughly the problem was the following:
When requesting all the sites we move the sites having a group into group rows. This is basically a structure like this:
When eg
offset 3, limit 2
was requested, only thesite 7
was returned as the offset was applied only on the first level. I tried different implementations and ways to solve this. I do now implement a custom limit filter that is applied after sort (generic filter) and before all others. Basically I turn this array into a flat structure like thisand then apply the filter. In the past logic we turned all the sites into such a flat array after all filters were run but we have to do it actually after the generic filters were run to make sure we do not apply all the queued filters and processed metrics on all rows.
I tested it with 33k sites and it was quite fast. Especially since people usually do not page through 100 pages.