Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DoNotTrack not recognized by matomo optout-script #12701

Closed
xopez opened this issue Apr 6, 2018 · 23 comments
Closed

DoNotTrack not recognized by matomo optout-script #12701

xopez opened this issue Apr 6, 2018 · 23 comments

Comments

@xopez
Copy link

xopez commented Apr 6, 2018

Hello everyone,

I noticed that on LineageOS (Android Rom) the default browser (built-in and I think its Jelly Browser) doesn't really recognize the DoNotTrack-function. I set the option in the browser and my page still says that I can Opt-Out. Other Browsers are working fine and I'm told I have the option set. Don't know it's a browser bug or not. Screens with German text, but the position where it stands is marked red.

android package name of the browser: org.lieangeos.jelly
useragent: Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36

OptOut-Iframe:
<iframe style="border: 0; height: 200px; width: 600px;" src="https://analytics.mightful-noobs.de/index.php?module=CoreAdminHome&amp;action=optOut&amp;language=de&amp;backgroundColor=&amp;fontColor=606060&amp;fontSize=100%&amp;fontFamily=Montserrat%20Regular" width="300" height="150"></iframe>

screenshot_20180406-131112
screenshot_20180406-131107

@sgiehl
Copy link
Member

sgiehl commented Apr 6, 2018

Are you able to figure out which headers your browser sends when DNT is enabled?

@xopez
Copy link
Author

xopez commented Apr 6, 2018

I can give you this with mod_log_forensic of my apache-server

+Wsdr8QUJi8sAACuXGHkAAABF|GET /piwik.php?ping=1&idsite=1&rec=1&r=618026&h=14&m=45&s=37&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018737&_idvc=1&_idn=1&_refts=0&_viewts=1523018737&send_image=1&cookie=1&res=458x813&gt_ms=170&pv_id=BjbJiy HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr8QUJi8sAACuXGHkAAABF
+Wsdr-QUJi8sAACuWX4MAAAAF|GET /datenschutzerklaerung/ HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|Upgrade-Insecure-Requests:1|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8|dnt:1|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuWX4MAAAAF
+Wsdr-QUJi8sAACuWX4QAAAAE|GET /wp-content/cache/wpfc-minified/342b2691b69150851c22ca32d736dd98/1523012503index.css HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/css,*/*;q=0.1|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuWX4QAAAAE
+Wsdr-QUJi8sAACuXGHoAAABJ|GET /wp-content/cache/wpfc-minified/368665ef6540fd2eb25ad7491c2ef378/1523013122index.js HTTP/1.1|Host:mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:*/*|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuXGHoAAABJ
+Wsdr-QUJi8sAACuXGHsAAABI|GET /index.php?module=CoreAdminHome&action=optOut&language=de&backgroundColor=&fontColor=606060&fontSize=100%25&fontFamily=Montserrat%2520Regular HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|Upgrade-Insecure-Requests:1|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-QUJi8sAACuXGHsAAABI
+Wsdr-gUJi8sAACuXGHwAAABL|GET /piwik.php?action_name=Datenschutzerkl%25C3%25A4rung%2520%257C%2520Mightful%2520Noobs&idsite=1&rec=1&r=691968&h=14&m=45&s=50&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018750&_idvc=1&_idn=1&_refts=0&_viewts=1523018750&send_image=1&cookie=0&res=458x813&gt_ms=84&pv_id=JoDFZe&webgl=1 HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-Wsdr-gUJi8sAACuXGHwAAABL
+WsdsAQUJi8sAACuXGH0AAABK|GET /piwik.php?ping=1&idsite=1&rec=1&r=116337&h=14&m=45&s=53&url=https%253A%252F%252Fmightful-noobs.de%252Fdatenschutzerklaerung%252F&_id=&_idts=1523018753&_idvc=1&_idn=1&_refts=0&_viewts=1523018753&send_image=1&cookie=1&res=458x813&gt_ms=170&pv_id=BjbJiy HTTP/1.1|Host:analytics.mightful-noobs.de|Connection:keep-alive|User-Agent:Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36|Accept:image/webp,image/apng,image/*,*/*;q=0.8|Referer:https%3a//mightful-noobs.de/datenschutzerklaerung/|Accept-Encoding:gzip, deflate|Accept-Language:de-DE,en-US;q=0.9|X-Requested-With:org.lineageos.jelly
-WsdsAQUJi8sAACuXGH0AAABK

LogFormat for this line in Apache:
LogFormat "%{forensic-id}n %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

As you can see "dnt:1" at 3rd line is send.

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

Hi, I can reproduce it with LineageOS 14.1 and org.lineageos.jelly.
I'll check to find out why.

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

I fail to setup SSL with mitmproxy so I can only intercept HTTP requests, but it seems like no matter how one sets the "Aktivitäten nicht verfolgen" setting, it never seems to add an dnt header.
grafik

Opening http://request.urih.com/ in the browser shows the same.

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

@xopez Which version are you using exactly?

I am having a different user agent then you

yours: Mozilla/5.0 (Linux; Android 7.1.2; HTC One A9 Build/NJH47F) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/65.0.3325.144 Mobile Safari/537.36
mine:  Mozilla/5.0  (Linux; Android 7.1.2; ONEPLUS A3003 Build/NJH47F) AppleWebKit/537.36  (KHTML, like Gecko) Version/4.0 Chrome/63.0.3239.111 Mobile  Safari/537.36

UPDATE:
That's fascinating: When I switch the WebView-Imprementation in the Android Developer Options from "AOSP WebView" (the open source LineageOS ones) to "Chrome Stable" it starts sending a Dnt:1 header and $_SERVER["HTTP_DNT"]=="1".

But the Opt-Out screen is still shown.

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

@sgiehl It seems like the request even gets tracked:
https://gist.github.com/Findus23/0913c69f10c4ff5ce2a0b53e8c98ef3e

@xopez
Copy link
Author

xopez commented Apr 6, 2018

I am using

14.1-20180405-NIGHTLY-hiaeuhl

But also noticed it before the version.
But I patch mostly every version and latest gapps-pico package to keep my device up2date.

@xopez
Copy link
Author

xopez commented Apr 6, 2018

yes, cause it says it didn't find DNT.
DEBUG PrivacyManager[2018-04-06 13:40:43 UTC] [92aa0] DoNotTrack header not found
https://gist.github.com/Findus23/0913c69f10c4ff5ce2a0b53e8c98ef3e#file-gistfile1-txt-L52

@fdellwing
Copy link
Contributor

As it works fine with Chrome, this is a Jellybug, isn't it?

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

@fdellwing At least partly.

But I'm stil unsure why it isn't working when one switches to the Chrome Webview, because it sends dnt:1 and $_SERVER["HTTP_DNT"] is correctly set to "1"

@fdellwing
Copy link
Contributor

I don't have Jelly installed, only Chrome, so I cannot help figuring this out.

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

Isn't it installed by default (simply called "Browser")?

@xopez
Copy link
Author

xopez commented Apr 6, 2018

yes. The jelly-browser is called "Browser" in the app launcher in LineageOS. And it's a system app, so can't remove it so easily.

@fdellwing
Copy link
Contributor

fdellwing commented Apr 6, 2018

I think I choose to get rid of it for Chrome while setting up the phone. At least there is no default Browser app on my phone anymore.

screenshot_20180406-172529

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

After a lot of debugging I have now finally found (at least part of) the solution:

I have created an HTML page with an opt-out-iframe of an non-https matomo instance (so I can proxy the request)

It turns out that the DNT-header isn't sent to pages in iFrames.
screenshot_20180406_174951

@xopez So if you could try out to directly surf to the https://yourmatomo.example/index.php?module=CoreAdminHome&action=optOut&idsite=14&language=de URL, it should correctly show the Opt-Out.

@fdellwing
Copy link
Contributor

It turns out that the DNT-header isn't sent to pages in iFrames.

So, a problem with the implementation of iframe in jelly?

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

And to fully solve the mystery and show that this has nothing to do with Matomo:

I added JS to the page to make a AJAX request, and it turns out that it also doesn't get a DNT header:

grafik

Therefore Matomo has no chance to know that the user has enabled DNT and therefore tracks the user.

I'll create a bugreport to LineageOS as it seems the DNT-feature is completely useless.

@Findus23 Findus23 closed this as completed Apr 6, 2018
@fdellwing
Copy link
Contributor

@Findus23 I dont see a thread in jira, can you provide a link?

@Findus23
Copy link
Member

Findus23 commented Apr 6, 2018

They only open their bug tracker from saturday to Sunday.

@fdellwing
Copy link
Contributor

Ok, if you are able to please send it to me via forum :)

@xopez
Copy link
Author

xopez commented Apr 6, 2018

Just post it here, so we can follow it.

@Findus23
Copy link
Member

Findus23 commented Apr 7, 2018

@fdellwing, @xopez
The bug has been reported here: https://jira.lineageos.org/browse/BUGBASH-1552
Also a bit unfortunately that a great OS encouraging people not to depend on Google are using Google Analytics on their website.

@Findus23
Copy link
Member

Findus23 commented Apr 8, 2018

Semi-related:
I have now started a discussion about using Google Analytics:
https://www.reddit.com/r/LineageOS/comments/8aowso/please_dont_use_google_analytics/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants