Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JS Tracker: Add possibility to turn cookies back on, enableCookies #13056

Closed
WouterFlorijn opened this issue Jun 11, 2018 · 29 comments
Closed

JS Tracker: Add possibility to turn cookies back on, enableCookies #13056

WouterFlorijn opened this issue Jun 11, 2018 · 29 comments
Assignees
Labels
c: Privacy For issues that impact or improve the privacy.
Milestone

Comments

@WouterFlorijn
Copy link

WouterFlorijn commented Jun 11, 2018

I'm working on a website that allows visitors to toggle cookies on or off. Currently, with Matomo I can turn them off using

window._paq.push(['disableCookies'])

I searched for a way to turn them back on, and tried

window._paq.push(['enableCookies'])

but that did not work.

Is there a way to turn cookies back on? If not, it would be great if it could be added.

Note from tsteur: Also need to look at this when working on it: #13056 (comment)

@sgiehl
Copy link
Member

sgiehl commented Jun 20, 2018

If the page is reloaded and disableCookies isn't pushed the cookies should be automatically used "again".
But for singe page apps it would make sense to have a method to turn the cookies back on.

@WouterFlorijn
Copy link
Author

@sgiehl That would be great. Not only for SPAs, but any website where the user can change their cookie consent settings on the fly, which is relevant with the GDPR.

@bjoe2k4
Copy link

bjoe2k4 commented Jun 9, 2019

Any Update on this? Interested in this as well!

@mattab mattab added the c: Privacy For issues that impact or improve the privacy. label Nov 5, 2019
@mattab mattab changed the title Add possibility to turn cookies back on Add possibility to turn cookies back on, JS enableCookies Nov 5, 2019
@mattab mattab added this to the 4.1.0 milestone Nov 5, 2019
@martin-braun
Copy link

martin-braun commented Feb 11, 2020

The biggest issue on this is not that we cannot re-enable cookies. As a bypass, we can simply reload and prevent pushing 'disableCookies'. However, if we use a custom opt-out form, it will not work after disabling cookies, because the opt-out form requires cookies to be enabled.

It would be great to be able to allow cookies by JS without reload and make it work, so the custom opt-out solution can work. In it's current design, it's impossible to prevent conventional Matomo cookies and still getting the Opt-Out cookie.

It's been 1 1/2 years, this needs to be addressed, to be honest.

@ctholho
Copy link

ctholho commented Apr 29, 2020

I'm also looking for this functionality. Is there any other way to reenable cookies without a reload?

I don't understand how this flies with GDPR and cookie consent regulation in the EU and other territories. I don't want to disable tracking entirely. Simply cookies on/off is enough, I would have thought.

@futureweb
Copy link
Contributor

Would also need this Feature, we use Matomo's disableCookies Feature + Klaro! as Consent Manager ... of course if User gives Consent I want to use Cookies again (1-Pager, ...), so it would be benefical if one could re-activate Cookies by JS

@utrautmann
Copy link

utrautmann commented Jun 18, 2020

If the page is reloaded and disableCookies isn't pushed the cookies should be automatically used "again".

That works technically. But in this case, however, a new visitor ID is generated in the cookie.
I.e. I now have 2 visitors (one for fingerprinting-scenario, another one for the cookie-scenario) in Matomo, although in reality it is only 1 visitor. That is not what you want.

@tsteur
Copy link
Member

tsteur commented Jun 18, 2020

@utrautmann had a quick look in the code. That's something we'll need to fix as I assume that's because we include whether the user has cookies or not in the fingerprint: https://github.com/matomo-org/matomo/blob/3.13.6/core/Tracker/Settings.php#L145

It should be an easy fix for us to remove this from the fingerprint when we work on it or fill this setting depending on navigator.cookiesEnabled setting.

@tsteur
Copy link
Member

tsteur commented Jun 18, 2020

Thanks for mentioning it and finding this 👍

@utrautmann
Copy link

@utrautmann had a quick look in the code. That's something we'll need to fix as I assume that's because we include whether the user has cookies or not in the fingerprint: https://github.com/matomo-org/matomo/blob/3.13.6/core/Tracker/Settings.php#L145

It should be an easy fix for us to remove this from the fingerprint when we work on it or fill this setting depending on navigator.cookiesEnabled setting.

@tsteur Is there an ticket where I can follow?

@tsteur
Copy link
Member

tsteur commented Jun 20, 2020

We'll fix this as part of this issue.

@utrautmann
Copy link

@tsteur
Oh it's late. Many of my customers would like to act in Germany after the BGH judgment and change the tracking. They want to go with fingerprinting as the default and setting cookies after the consent.
Is there a way to still release that in the 3.13.x product line?
The alternative would be to patch the PHP file for yourself in every installation!?

@tsteur
Copy link
Member

tsteur commented Jun 22, 2020

@utrautmann I was more thinking something like this instead of ignoring the cookie server side: https://github.com/matomo-org/matomo/pull/16101/files

Are you familiar with patching files on a server? If so you could try to update your piwik.js, matomo.js, and js/piwik.min.js with this content and then give it a test: https://raw.githubusercontent.com/matomo-org/matomo/8ad6c25f3d1bb651444aff2ddf492c041dcb7306/matomo.js

@utrautmann
Copy link

@tsteur Thank you. I patched the files on my server. Now I have tested with different browsers.
It works fine with Firefox, Chrome, Chrome-Mobile, Edge, Opera. But not with IE11.
Two different visitor IDs are still created with IE11.

@tsteur
Copy link
Member

tsteur commented Jun 24, 2020

Thanks for confirming. IE11 is because of #11507

It seems we can't easily check if cookies are enabled without actually setting a cookie there or causing a warning. Unless we could test if the site is shown in a modal dialog or not but not sure how/if that's possible. If you have any thoughts let me know

@tsteur
Copy link
Member

tsteur commented Jun 25, 2020

Hi there, anyone keen on giving the enableCookies method a try? #16113

If you are familiar with patching files on a server then you could try to update your piwik.js, matomo.js, and js/piwik.min.js with the content from the following URL and then give it a try: https://raw.githubusercontent.com/matomo-org/matomo/567ca3070a5ea62660d347b53d3704258af4f59c/matomo.js

I did some basic testing and it works for me.

Similar to disableCookies it includes a new method _paq.push(['enableCookies']);.

@utrautmann @futureweb @ctholho @martin-braun @WouterFlorijn @bjoe2k4

@utrautmann
Copy link

Hi there, anyone keen on giving the enableCookies method a try?

I tested it with enableCookies. It works fine in single page websites. Thanks alot.

  1. EnableCookies creates a cookie _pk_id ... and
  2. disableCookies removes the Matomo cookies.
  3. When I trigger an event after EnableCookies I get the cookie _pk_ses... .
  4. But I never got a _pk_ref... Cookie for attribution. If I call up a page (single page or normal page) with parameter pk_campaign and do not have the user's consent at the moment and then get it and call enableCookies, no cookie _pk_ref is set.
    Is this possible so that I can measure the attribution in goals?

To IE11:
In a single page all actions of a visitor - started with disabled cookies and go on with enabled cookies - are always assigned to the same visitor as long as I don't reload the single page website. Then I have the IE11 problem from above again, that one and the same visitor counts as 2 visitors depending without and with cookies.

@tsteur
Copy link
Member

tsteur commented Jun 26, 2020

The IE 11 is kind of expected and can't do really much about it except disabling the cookie as part of the fingerprint on the server itself which I now did in this PR: https://github.com/matomo-org/matomo/pull/16113/files#diff-02edccc593e3badd1e9fd8d278fd1d35R102-R147

This will then make it work.

@tsteur tsteur self-assigned this Jun 26, 2020
@utrautmann
Copy link

utrautmann commented Jun 26, 2020

@tsteur

This will then make it work.

Thank you very much. I've tested it successful with IE11.

Could you take a look to my question number 4 please?

But I never got a _pk_ref... Cookie for attribution. If I call up a page (single page or normal page) with parameter pk_campaign and do not have the user's consent at the moment and then get it and call enableCookies, no cookie _pk_ref is set.
Is this possible so that I can measure the attribution in goals?

@tsteur
Copy link
Member

tsteur commented Jun 26, 2020

Thanks @utrautmann I've made a minor adjustment but haven't tested it yet. I reckon it should be set now. Seems so far Matomo sets attribution cookie only when a tracking request is done so when you enable cookies and there's no tracking request it wouldn't be set.

@tsteur
Copy link
Member

tsteur commented Jun 29, 2020

FYI @utrautmann did some more testing and the referrer cookie is now set.

However, there's one thing that's not quite clear...

I was going to send a tracking request to update the existing visit with the newly created visitorId when enabling cookies. Noticed that Matomo does currently not support updating the visitorId of an existing visit.

What this means is the next time this visitor visits your website, it will be counted as a new unique visitor and only then will be tracked with the new visitorId. So flow is like this

  • Cookies are disabled.
    • Matomo identifies this visitor by a fingerprint.
    • On the backend a random visitorId is used which is not known to JavaScript.
  • Now user gives consent and you enable cookies.
    • Matomo generates a random visitorId in JS because it can't know which visitorId was created server side
    • However, the current visit still has a different visitorId because we can't overwrite it
  • Visitor leaves your website
  • Visitor visits your website again say one hour or a day later.
    • This visit will be recorded under the new idVisitor.
    • Also all following visits in the future will be recorded under this new visitorId

So it basically once create a new unique visitor and in total Matomo counts 2 unique visitors.

I was thinking about creating some functionality to update the visitorId when the user gives consent and we enables cookies. This way it would have counted only one unique visitor. However, I'm not sure if we can/should do this since we maybe didn't have consent to attribute all previous page views before the visitor gave consent to this unique visitor. I'm meaning the page views that were done within the visit in which the user enables the cookies and gives consent:

  • Cookies are disabled.
    • Matomo identifies this visitor by a fingerprint.
    • On the backend a random visitorId is used which is not known to JavaScript.
  • Now user gives consent and you enable cookies.
    • Matomo generates a random visitorId in JS because it can't know which visitorId was created server side
    • We send a tracking request and attribute all previous page views from this visit to this new visitorId
    • Basically, all previous page views before the user gave consent would be attributed to a unique visitor but back then we didn't have consent.

I don't think we can/should do this. I guess if this behaviour was wanted, someone should maybe force a new visit so the visits before giving consent and after giving consent are clearly separated.

I can see some would maybe want to create a new visit while some others maybe wouldn't want to.

Anyone any thoughts on this?

It's quite complicated so I hope everyone can follow my logic.

@utrautmann
Copy link

Now user gives consent and you enable cookies.
Matomo generates a random visitorId in JS because it can't know which visitorId was created server side
However, the current visit still has a different visitorId because we can't overwrite it
Visitor leaves your website
Visitor visits your website again say one hour or a day later.
This visit will be recorded under the new idVisitor.

Why is that? I thought after the enabling cookies I have a cookie with a visitor ID that is valid for 13 months and is recognized again?
Or will I be after enabling cookies further identified via backend fingerprinting in the visit?

I was thinking about creating some functionality to update the visitorId when the user gives consent and we enables cookies.

I understand the problem you are describing.
I thought that consent is only required for tracking with cookies. However, but it's no consent for cookie-less tracking required.
Or do you see a consent for cookie-less tracking as necessary?

In my opinion, it is not critical from a data protection point of view if the page views from the current visit with cookie-less tracking are attributed to the same visitor after the consent to tracking with cookies.

I can see some would maybe want to create a new visit while some others maybe wouldn't want to.

In that case, an option would be required, depending on the customer's requirements, right?

@tsteur
Copy link
Member

tsteur commented Jun 29, 2020

Why is that? I thought after the enabling cookies I have a cookie with a visitor ID that is valid for 13 months and is recognized again?
Or will I be after enabling cookies further identified via backend fingerprinting in the visit?

That's right the cookie will be set immediately for 13 months and is valid for that long. It's just that for the already started visit Matomo knows this visit under a different visitorId which is only known in the Matomo backend but can't be known in the Matomo JavaScript tracker. And when Matomo receives the new visitorId from the JS tracker it is currently not updating the visitorId of the already started visit. It would become only active on the next visit.

Or do you see a consent for cookie-less tracking as necessary?

As long as no personal data isn't tracked there shouldn't be any consent needed. See eg this FAQ.

I was just wondering because technically the previous page views (might be 1 or might be 10...) while in the current visit cookies were disabled maybe weren't meant to be put into the visitor's profile that is being created once consent is granted. Considering this would only affect the page views within the current visit it might be fine though.

@utrautmann
Copy link

So it basically once create a new unique visitor and in total Matomo counts 2 unique visitors.

I tested that again after a day. After my first visit (disable and then enable cookies) I am back on my test page. The visitor log says that my visit is a returning visitor. That would be good. But the visitor profile says there are no more visits for this visitor. It's a little strange. How does Matomo know that I returned? Is it from the one day old cookie? But Matomo cannot find the VisitorID from these cookies anywhere, because my first visit had a Backend-VisitorID? Is that correct?

I was just wondering because technically the previous page views (might be 1 or might be 10...) while in the current visit cookies were disabled maybe weren't meant to be put into the visitor's profile that is being created once consent is granted. Considering this would only affect the page views within the current visit it might be fine though.

Ok, now I understand your thoughts. Well, I don't think that's so important to most web analysts to see the pages in new visitors profile. They look at the sums of visitors and page views.
But there could be things where it would be interesting, right?
Just one thought from me: I have a customer who works with the Funnel plugin. The entry URLs could be wrong if the page on which the consent is given is missing in the profile. And could the page where i'm giving the consent be an exit page? Or am I wrong and that's not a problem with entry and exit pages?

@tsteur
Copy link
Member

tsteur commented Jul 2, 2020

@utrautmann I've just tweaked the PR so the existing visit will be updated and it will be only one unique visitor in the end. So things should all work then.

@utrautmann
Copy link

@tsteur : Thank you very much. What are the schedules for this PR? I ask if my customer should wait for it or if the server should be patched.

@tsteur
Copy link
Member

tsteur commented Jul 2, 2020

I would say in 2 weeks

@tsteur
Copy link
Member

tsteur commented Jul 6, 2020

@utrautmann sorry to let you know but I renamed enableCookies to setCookieConsentGiven in this PR #16178 while documenting this new feature in https://github.com/matomo-org/developer-documentation/pull/359/files

This will be renamed to be consistent in how we ask for tracking consent. Asking for cookie consent works now in a very same way and is described in both PRs above.

There will now also be a new method rememberCookieConsentGiven which makes cookie consent really easy as all you need to do is always calling

_paq.push(['requireCookieConsent'])

instead of disableCookies and then once the user gives consent, you only need to once call _paq.push(['rememberCookieConsentGiven']) to let Matomo know to use cookies (also in the next visits). When user no longer wants to use cookies simply call _paq.push(['forgetCookieConsentGiven'])

@tsteur tsteur closed this as completed Jul 6, 2020
@tsteur
Copy link
Member

tsteur commented Jul 6, 2020

I'm hoping we can release a beta this week.

@mattab mattab modified the milestones: 4.1.0, 4.0.0 Sep 9, 2020
@mattab mattab changed the title Add possibility to turn cookies back on, JS enableCookies JS Tracker: Add possibility to turn cookies back on, enableCookies Sep 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Privacy For issues that impact or improve the privacy.
Projects
None yet
Development

No branches or pull requests

9 participants