Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide features that don't work when the website is configured in cookie-less tracking #16363

Open
mattab opened this issue Sep 1, 2020 · 13 comments · May be fixed by #16773
Open

Hide features that don't work when the website is configured in cookie-less tracking #16363

mattab opened this issue Sep 1, 2020 · 13 comments · May be fixed by #16773
Labels
c: Consistent Reports & Analytics UX For bugs and features that make Analytics reporting UI behave more consistently. c: Data Integrity & Accuracy c: Privacy For issues that impact or improve the privacy. c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.

Comments

@mattab
Copy link
Member

mattab commented Sep 1, 2020

More and more people are disabling tracking cookies (we sometimes call it cookie-less tracking), because no cookie consent is needed then.

When cookie-less tracking is enabled some features don't work for example:

  • Unique Visitors metric is very inaccurate
  • New VS Returning visitors
  • Goals and Ecommerce conversions will be attributed to the channel used in the visit that converts
  • Some reports are very inaccurate:
    • Days since last visit,
    • Visits by visit count,
    • Visits to Conversion,
    • Days to Conversion
    • Cohorts
    • Multi Attribution

Because the features become very broken after cookie-less is enabled, people tend to get confused, and ask questions, or they may use the data not knowing it's very wrong! So it would be really useful to somehow hide these reports from users, or make it otherwise crystal clear that they shouldn't use this data. What do you think?

Note: we'd leave Visitor profile as it's helpful when user id is used

required for #15507

@mattab mattab added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. c: Usability For issues that let users achieve a defined goal more effectively or efficiently. labels Sep 1, 2020
@tsteur tsteur added the c: Privacy For issues that impact or improve the privacy. label Sep 3, 2020
@tsteur
Copy link
Member

tsteur commented Sep 3, 2020

Hiding this data was also suggested in #13655 (comment) but we didn't do it since it can currently not be enforced and is only done client side. There are few more features like Cohorts, Multi Attribution, Visitor Profile, etc. Hiding this features might only be a good idea though if not also the userId feature is used.

This feature will only work when cookies can be disabled server side eg see #16258 . We could look at making this work after #16258 was developed

@tsteur
Copy link
Member

tsteur commented Oct 26, 2020

Here's what @mattab and I discussed:

We archive a new metric "nb_profilable". It is the sum of how many people had cookies (or similar technology such as advertising identifier on iOS etc) enabled to identify them over several visits. If 99% or more of the visits were not "profilable" (had cookies disabled), then we assume the site is tracking cookie-less and remove certain features etc see issue description.

It basically means for every requested date range we probably also need to check if there were "profilable" visits or not.

We don't need to expose the new metrics to the UI for now but could do this later. This can be quite useful later to be able to see what percentage basically consented to cookies etc.

@mattab I just looked in the code and log_visit.profilable=1 also when userId is used. This kind of makes sense because a profile for that visitor is possible. So in the small percentage where users use the userId feature then we would not disable the features mentioned in the issue description. Not sure if this would be an issue or if we would want to workaround it?

@mattab
Copy link
Member Author

mattab commented Oct 28, 2020

just looked in the code and log_visit.profilable=1 also when userId is used. This kind of makes sense because a profile for that visitor is possible. So in the small percentage where users use the userId feature then we would not disable the features mentioned in the issue description. Not sure if this would be an issue or if we would want to workaround it?

Good question... Btw as I'm not sure anymore how the field profilable is used by Matomo. could we document the field in https://developer.matomo.org/guides/persistence-and-the-mysql-backend and maybe add an entry in glossary for Profilable https://glossary.matomo.org/ (or a FAQ if there's enough material but probably not)?

if we were to workaround, what would be the options?

(eg. would it make sense to have a separate column set to 1 when visitors had tracking cookies activated?)

@tsteur
Copy link
Member

tsteur commented Oct 28, 2020

faq is eg here: https://matomo.org/faq/how-to/how-do-i-segment-people-that-have-been-identified-using-cookies-vs-fingerprint/

Profilable means you can create a profile of a user. That's either through cookies, userId or a fixed set visitorId.

Workaround means select sum() where profilable = 1 and user_id is null or something... It wouldn't be 100% accurate maybe since a user maybe has cookies plus user_id (the user_id feature was implemented wrong from the beginning so it's hard to separate). I guess it might be edge case though as only small percentage of our users use user_id and technically user_id would need consent anyway so you can assume in most cases cookies were used.

Let's keep it simple and simply sum how many users had profilable=0 and when 99% or more of visits were not profilable then we disable all the features.

@diosmosis diosmosis self-assigned this Oct 29, 2020
@diosmosis
Copy link
Member

Couple questions:

  • should metrics be disabled in the API as well? or just the UI?
  • should segments (ie, visitorType) be disabled (ie, removed)?

@mattab
Copy link
Member Author

mattab commented Nov 1, 2020

should metrics be disabled in the API as well? or just the UI?

Since we remove metrics from the UI, we would also ideally remove them from the API (so it's consistent with the UI, and also so that the data shown in API is trusted and it's good to know it doesn't return invalid data).

Also could you keep track of which metrics are affected and compile them in a list, so you can publish the list of API metrics/reports affected in the existing FAQ: https://matomo.org/faq/general/faq_156/

should segments (ie, visitorType) be disabled (ie, removed)?

When a segment is not compatible with the current site, then we could "hide" the segment from the list in the UI (but in the Segment APIs we would still return all segments).

Also we want to make sure data for any segment on any website is not archived if the segment is invalid for this website.

@mattab
Copy link
Member Author

mattab commented Nov 1, 2020

Regarding hiding segments, it would be great to also add a section in https://developer.matomo.org/api-reference/reporting-api-segmentation for example List of segments that require tracking cookies where we explain some segments are hidden and list the segments that require cookies.

and ideally we also mention in the user guide https://matomo.org/docs/segmentation/ that some segments require tracking cookie and link to the list on the developer page.

@tsteur
Copy link
Member

tsteur commented Nov 9, 2020

@diosmosis do you think we can finish this issue ideally this week so it can make it into the release?

@diosmosis
Copy link
Member

@tsteur I'll give it a shot.

@tsteur
Copy link
Member

tsteur commented Nov 12, 2020

@diosmosis I'll move this for now to Matomo 4.1 milestone. The other issues have currently priority and we can always release it later. Feel free to continue working on it and we could otherwise still see if it can make it into 4.0 release or not

fyi @mattab

@sgiehl
Copy link
Member

sgiehl commented Feb 14, 2022

@diosmosis Will you still be working on this one? I guess the PR is still waiting on feedback from @mattab. Was just wondering if we should unassign you, so someone else might take over from here on.

@diosmosis diosmosis removed their assignment Feb 14, 2022
@diosmosis
Copy link
Member

@sgiehl yes, just unassigned myself

@mattab
Copy link
Member Author

mattab commented Dec 11, 2023

Note: a PR was started in #16773 (Thanks @diosmosis )
and this then stalled due to lack of review from me at the time.
Anybody picking this up later, let's discuss here the feedback 💡

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Consistent Reports & Analytics UX For bugs and features that make Analytics reporting UI behave more consistently. c: Data Integrity & Accuracy c: Privacy For issues that impact or improve the privacy. c: Usability For issues that let users achieve a defined goal more effectively or efficiently. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants