Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fast close request and continue running php #10093

Closed
wants to merge 7 commits into from

Conversation

joanhey
Copy link

@joanhey joanhey commented Apr 25, 2016

Piwik will not be faster, but perceived performance will be. Website speed with javascript client using this file will be only the latency.

Piwik will not be faster, but perceived performance will be. Website speed with javascript client using this file will be only the latency.
@joanhey
Copy link
Author

joanhey commented Apr 25, 2016

I was bored that piwik.php was ~1/3 of the total website load speed. It's asynchronous load, but better now.

Before

piwik-before

After

piwik-after

Only works with php-fpm and mod_php, more than 90% of servers, and using the validation will be no problem with other SAPI.

I created a new file for use with the javascript client in websites, so the apps using the API can continue using the normal piwik.php. We only need to show to the website visitors the 204 No response (not 400 Bad Request, ...). Also now don't send Content-type (not necessary for status 204).

Tested with and without cookies without problems.

If accepted, will be necessary to change the creation of the javascript code created in the config for add to the websites. Directly or validating the SAPI.

Thanks for the wonderful Piwik.

@tsteur
Copy link
Member

tsteur commented Apr 26, 2016

We do have eg bulk tracking where we return a response in json format like how many requests were valid, invalid, ... for this a response is kind of needed. Via the API send_image=1/0 it is also possible to request an image response (if not bulk tracking, then response is always json). Lots of our tests depend on the correct return code and we use the return code etc to check whether the data was inserted correctly etc.

We could maybe integrate this feature into piwik.php via a separate query parameter like immediate_response=1/0 or something. In the JS tracker we could add an API for this as well so users can test it and once we can confirm it works we can see if we can enable it automatically at some point. However, we could send the response only after parsing the GET/POST parameters so that we know whether we are dealing with bulk tracking etc.

@mattab
Copy link
Member

mattab commented May 27, 2016

@tsteur interestingly none of the tests are failing with this change - I wonder if some tests should have failed...?

@mattab mattab added the Needs Review PRs that need a code review label May 27, 2016
@mattab mattab added this to the 2.16.x (LTS) milestone May 27, 2016
@tsteur
Copy link
Member

tsteur commented May 27, 2016

It's a different file, nothing can fail

@joanhey
Copy link
Author

joanhey commented May 27, 2016

In my websites I use: _paq.push(["setTrackerUrl", u+"webpiwik.php"]);

without problems.

@joanhey
Copy link
Author

joanhey commented May 27, 2016

We must distinguish between app tracking and normal website page visits.

If we move it to a query parameter, we are moving the solution in to the problem.

In normal websites we don't make bulk tracking, no POST,...
The people include: http://developer.piwik.org/guides/tracking-javascript-guide
Also our website visitors don't need to see: 400 Bad request, JSON responses,...

I created a new file, because it's easier to see what do exactly, don't break and we can make changes and tests.
So now, we have another endpoint for the api. We don't change the actual endpoints.

If you prefer, later we can include it, in the piwik.php.
But so will change also de client, and will use the same endpoint (piwik.php).
We need the client request with HEAD method and not GET, when we want only the 204 response.

Also a good candidate for this optimization, it is when we req an image. We can send the 1x1 gif, close connection and continue processing the data.
When we request an image, we aren't sending bulk data, we don't send POST, we don't wait for json response, ... only the image, and collect the visitors data.

You will choose if you prefer a new endpoint, or change the method to HEAD ( or image content-type for images) when you only need collect the data.

This change don't break anything for app tracking, only add a new endpoint for tracking website page visits. When you want bulk tracking or POST don't send it to this endpoint.

If you need more info, only ask me.

Thank you

@mattab
Copy link
Member

mattab commented May 27, 2016

We could maybe integrate this feature into piwik.php via a separate query parameter like immediate_response=1/0 or something. In the JS tracker we could add an API for this as well so users can test it and once we can confirm it works we can see if we can enable it automatically at some point. However, we could send the response only after parsing the GET/POST parameters so that we know whether we are dealing with bulk tracking etc.

👍 @joanhey if we want to integrate this change in Piwik it must be done in the main tracking API endpoint as we can only have one API endpoint: piwik.php. Maybe you'd be interested to give it a try to implement this feature according to the above suggestion?

@joanhey
Copy link
Author

joanhey commented Jun 2, 2016

Included in piwik.php.

But not with a query parameter, or we move the solution into the problem.

When a client need only the "204 No Response", call to piwik.php with 'HEAD' not 'GET' method.

@mattab mattab added the c: Performance For when we could improve the performance / speed of Matomo. label Jul 18, 2016
@mattab mattab modified the milestones: 3.0.0, 2.16.x (LTS) Jul 18, 2016
@@ -16,6 +16,21 @@
// Note: if you wish to debug the Tracking API please see this documentation:
// http://developer.piwik.org/api-reference/tracking-api#debugging-the-tracker

if ($_SERVER['REQUEST_METHOD'] === 'HEAD') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$_SERVER['REQUEST_METHOD'] may not always be assigned I think. Probably it is but just to be sure I would maybe add a !empty($...)

@tsteur
Copy link
Member

tsteur commented Aug 30, 2016

We certainly need to have a URL parameter to enable this when wanted instead of HEAD. We would need to change the JS tracker to send a HEAD instead of a GET/POST which is kind of breaking API. Some servers might have HEAD requests disabled, gets blocked by firewall or something else so we cannot easily change this in piwik.js to HEAD.

Also I think HEAD is supposed to return identical response as GET would but without body which it does not. GET would return an HTTP 400 etc under circumstances.

We should check for a URL parameter and we could have an opt in for this in piwik.js in the beginning and we get good feedback and if there are no problems (when problems are fixed) make it the default behaviour for piwik.js.

As Piwik 2 aka master branch is in LTS mode and we only merge critical fixes can you issue this PR against the "3.x-dev" branch?

@tsteur tsteur changed the base branch from master to 2.x-dev September 1, 2016 06:38
@mattab
Copy link
Member

mattab commented Sep 23, 2016

Hi @joanhey
Thanks for the PR. Do you think you will be able to implement the changes that @tsteur suggested in the comment above (his code review) ?

We'd be happy to consider merging this if you finish the PR, otherwise we will have to close it. Thanks!

@mattab mattab added the Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. label Sep 23, 2016
@mattab mattab modified the milestones: 3.0.0-b2, 3.0.0 Sep 23, 2016
@joanhey
Copy link
Author

joanhey commented Oct 12, 2016

The next month I'll have time to make the changes

@mattab mattab modified the milestones: 3.0.0-b2, 3.0.0-b3 Oct 30, 2016
@mattab mattab modified the milestones: 3.0.0-b3, 3.0.0-b2 Oct 30, 2016
@mattab mattab modified the milestones: 3.0.0-b4, 3.0.0-b3 Nov 14, 2016
@mattab
Copy link
Member

mattab commented Nov 14, 2016

@joanhey Sounds good let us know how you go

@mattab mattab modified the milestones: 3.0.0, 3.0.0-b4 Nov 30, 2016
@mattab
Copy link
Member

mattab commented Apr 26, 2017

Thank you for this proposed pull request.

Because it was last updated more than one month ago, it is our policy to close pull requests opened for a long time without updates. If you would like to continue work on the pull request, please simply ping us to have it re-opened (after you have pushed a new commit).

We hope you understand this and we look forward to seeing an update from you on this pull request or another one!

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo. Needs Review PRs that need a code review Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants