Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handling of POST tracking requests when page is unloaded (using sendBeacon) #12538

Merged
merged 10 commits into from Mar 22, 2018

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented Feb 5, 2018

refs #6641

Anyone keen on doing some more testing with this?

Basically, what I experienced is, that we eg send a POST request on a form submit, then an unload happens and all pending requests are directly cancelled without hitting the Matomo tracking server, then we fall back to GET and those GET requests might fail because the URL is too long etc.

What would now happen here is when such a POST request is cancelled, and the unload was triggered, we would send them again as POST request which makes it more likely to succeed vs GET but this time using sendBeacon.

Also if any tracker plugin sends any tracking requests via unload plugin event, those tracking requests would be directly sent via sentBeacon as well vs before they likely did not finish on time before the next page was loaded.

@mattab mattab added this to the 3.3.1 milestone Feb 5, 2018
@mattab mattab added the Needs Review PRs that need a code review label Feb 5, 2018
@mattab
Copy link
Member

mattab commented Feb 5, 2018

FYI Haven't tested the code but browser compatibility looks pretty good now:

compat1
mobile

After reading the doc of sendBeacon it looks like the ideal solution for what we're doing! this should resolve some of the web server errors (at least on apache)

@tsteur
Copy link
Member Author

tsteur commented Feb 8, 2018

@mattab I tested it again and also checked in the logs and it worked.

There is one thing that is a bit annoying but it always has been like this already: I noticed sometimes 2 POST requests. Eg a user submits a form, a POST request is sent, a few ms later the "unload" event occurs and the browser cancels these requests, then we retry them and send the second request. In the callback where we usually fallback to get request, we cannot know if the initial POST was already successfully transferred to the server or not (eg maybe request completed but response wasn't sent yet) and in the worst case this could cause the same data to be tracked twice. This affects all POST requests that are made shortly before unload. I wonder if we should always execute such requests with like a 50ms timeout for example?

In my tests this prevent this problem like this:

                if (isPageUnloading && sendPostRequestViaSendBeacon(request)) {
                    return;
                }

                setTimeout(function () {
                    if (isPageUnloading && sendPostRequestViaSendBeacon(request)) {
   // meanwhile maybe the unload event occured
                        return;
                    }
                    sendRequestTheRegularPostWay();
                }, 50);

@tsteur
Copy link
Member Author

tsteur commented Feb 8, 2018

FYI: I believe this timeout I just explained is the way to go and helpful... therefore pushed the change.

@tsteur tsteur changed the title Use sendBeacon if available Better handling of POST tracking requests when page is unloaded (using sendBeacon) Feb 8, 2018
js/piwik.js Outdated

try {
var blob = new Blob([request], headers);
success = navigator.sendBeacon(configTrackerUrl, blob);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When sending the beacon, currently we don't get the data in the GET, but as POST. since it's useful to get GET requests so we can replay the logs, wondering if we should maybe do the same as in getImage and write eg. navigator.sendBeacon( configTrackerUrl + (configTrackerUrl.indexOf('?') < 0 ? '?' : '&') + request, blob) and send the data as GET params instead of POST?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mattab see #6641 (comment)

It is not possible to send a GET request but mentioned in #6641 we need to check if log importer can process POST request when URL parameters are used

@mattab
Copy link
Member

mattab commented Feb 9, 2018

Great find with the new setTimeout and 50 ms which will help track more accurate data 👍

@mattab mattab mentioned this pull request Mar 4, 2018
@mattab mattab self-assigned this Mar 22, 2018
@mattab mattab merged commit f9e283f into 3.x-dev Mar 22, 2018
@sgiehl sgiehl deleted the 6641 branch April 30, 2018 08:18
InfinityVoid pushed a commit to InfinityVoid/matomo that referenced this pull request Oct 11, 2018
…g sendBeacon) (matomo-org#12538)

* use sendBeacon if available

* better sendBeacon implementation

* execute post request with a little delay

* make sure to use alias so variable will be minimized

* trigger build

* Merged 3.x-dev

* Revert "Merged 3.x-dev"

This reverts commit 51addf1

* Minified files up to date
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants