Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send bulk requests in chunks when needed #13444

Merged
merged 3 commits into from Dec 6, 2018
Merged

Send bulk requests in chunks when needed #13444

merged 3 commits into from Dec 6, 2018

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Sep 18, 2018

fix #13116

It groups max 50 requests into one bulk requests and issues multiple requests if needed.

Assuming a request line has about 500 bytes, we are looking at around max 25KB... 25KB for a POST requests really shouldn't be any huge. Even 100K or 1MB shouldn't be any problem in theory. PHP default is 8MB.

We could also make things more complicated and limit by number of characters but hoping it is not needed.

@tsteur tsteur added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Needs Review PRs that need a code review labels Sep 18, 2018
@tsteur tsteur added this to the 3.7.0 milestone Sep 18, 2018
@diosmosis diosmosis merged commit 17f16ed into 3.x-dev Dec 6, 2018
@diosmosis diosmosis deleted the 13116 branch December 6, 2018 00:30
sgiehl pushed a commit that referenced this pull request Dec 6, 2018
* send bulk requests in chunks

* send requests correctly
diosmosis pushed a commit that referenced this pull request Dec 8, 2018
* Add reports dimensions to metadata of report and rows

* translate dimension columns

* updates test files

* fix possible error when no report is available

* update tests

* Improve subdimension detection

* Adjust tests for labelX logic

* Makes flattener compatible with 3 dimensions

* Adds new method getThirdLeveltableDimension to report class

* Do not ask for 2fa authentication code when CoreUpdater is being requested (#13796)

Could fix an edge case where user is logged in, but hasn't confirmed the auth code (so the user is not actually logged in), and then an update appears.

* Added Fallback Method for Alexa in SEO Plugin (#13552)

* added fallback method for Alexa, fixes issue #13427

* do not use short array syntax for consistency with other methods

* use mini link for Alexa, use DomXPath to filter out the global ranking instead of regex

* Use db sessions by default, deprecate file session handler (#13540)

* use db sessions by default, deprecate file session handler

* trying to fix tests

* Prevent trigger errors on demand for instances that are opened to anonymous (#13535)

fix #13513

* Remove the previous exception in base validator so the same error is not printed twice (#13801)

* Fixing build  (#13802)

* update submodule

* Update screenshots and try to get test to pass.

* Get SingleMetricView to pass. (#13803)

* Quickform2 throws warnings with PHP7.2 (#13463)

fixes #13272

Haven't actually tested it but should fix the issue. If tests pass, the logic would be still the same. I don't have a PHP 7.2 running here otherwise at the moment

* Send bulk requests in chunks when needed (#13444)

* send bulk requests in chunks

* send requests correctly

* Make log and report data screen less technical (#13464)

* When you are logged out, the URL gets lost when you log in (#13441)

It won't remember any hash as the hash won't be visible in the referrer etc but it would work for most other pages.

To make it work for hash it would get likely way more complicated like we would need to persist it through JS, temporarily store it somewhere and redirect accordingly. It fixes the case mentioned in the issue.

fix #13328

* show full information of URL only on extra click (#13585)

* Add option to opt in to use send beacon (#13451)

* Add option to opt in to use send beacon

* Fix JS tracker test.

* do not overrwite existing subrow metadata

* update test files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Content Tracking may cause huge POST requests to be sent (50Kb or more)
2 participants