Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove special handling of browsers with DNT enabled by default #13686

Merged
merged 4 commits into from Apr 11, 2019

Conversation

Findus23
Copy link
Member

@Findus23 Findus23 commented Nov 8, 2018

fixes #9987
also fixes #3531
Now that we advertise that Matomo respects DNT, we should do this as precisely as possible.

Of course this makes Matomo less accurate (as now IE11 users that never enabled DNT, but had it enabled by default are not tracked anymore), but I think those browsers are becoming less relevant in the future and removing the special handling is easier than explaining the technical details to all Matomo users.

A move verbose compromise solution would be having an option or plugin that restores this behavior.

@Findus23 Findus23 added c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Needs Review PRs that need a code review labels Nov 8, 2018
@Findus23 Findus23 added this to the 3.8.0 milestone Nov 8, 2018
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sgiehl
Copy link
Member

sgiehl commented Nov 10, 2018

We maybe should mention that in a changelog

@mattab
Copy link
Member

mattab commented Nov 14, 2018

Suggested steps:

  • Mention in developer changelog the change
  • When DNT is activated, in the browser report footer, show a message explaining that Did you know? Internet Explorer 11 and newer are mostly not tracked because they have DoNotTrack activated by default and Matomo <a href=privacy-settings>is configured to respect DoNotTrack</a>
  • Update the DNT inline help in the Privacy page and explain that some browsers will mostly not be tracked as they enable DoNotTrack by default and list them eg. "IE11 and newer"

@tsteur
Copy link
Member

tsteur commented Nov 14, 2018

FYI It shouldn't be mentioned in the developer changelog I would say. It's not related to the platform.

@tsteur
Copy link
Member

tsteur commented Nov 14, 2018

Also IE usage is nowadays at around 3% and maybe doesn't need to be mentioned in the browser report? And maybe only in the help text instead of footer to keep UI nice and clean and not clutter it with too unimportant things? Probably way more users are excluded with ad blockers etc

@sgiehl
Copy link
Member

sgiehl commented Nov 14, 2018

Also IE usage is nowadays at around 3%

That strongly depends on the website. Company intranets for example might have almost 100% IE in some cases. And especially those customers might need to change the Matomo config, as before the DNT setting didn't have any effect

@tsteur
Copy link
Member

tsteur commented Nov 14, 2018

Maybe we could show that when people are using the type IntranetWebsite? Not many would use that type just yet but eventually will. Or maybe could look at whether 2018 yearly archive had eg at least 5% IE users and then could show it. It'd be just annoying to show it to everyone all the time when it's not really that valuable for 98% of the users. You can't show messages for every edge cases otherwise it ends up quite funny :)

@fdellwing
Copy link
Contributor

That strongly depends on the website. Company intranets for example might have almost 100% IE in some cases. And especially those customers might need to change the Matomo config, as before the DNT setting didn't have any effect

We provide business software for hospitals and have >66% IE11 users.

@tsteur
Copy link
Member

tsteur commented Dec 3, 2018

I suppose best be to disable DNT then. For sure we shouldn't show the footer message to all users, only when IE 11 used to make a significant amount of traffic. Maybe someone could also develop a simple plugin to keep ignoring DNT for IE11 but not others

@Findus23
Copy link
Member Author

Findus23 commented Dec 3, 2018

@tsteur I think adding a plugin is the best way to handle the cases where the feature is needed (this way it is expected and users know the implications)

Would it work to post an event like

Piwik::postEvent('PrivacyManager.handleDNT', array($???));

here where plugins would get the user agent and detected DNT and then could return a boolean if the user should be tracked or not. (not sure if that is a bit too specific and if there is a more general way to make a plugin possible)
https://github.com/matomo-org/matomo/pull/13686/files#diff-698d1d7fe9c92e8a27eda0e8c4a88dedR75

@tsteur
Copy link
Member

tsteur commented Dec 4, 2018

Yes such an event would work. Eg postEvent('PrivacyManager.shouldIgnoreDnt', array(&$shouldIgnore = false));

@sgiehl
Copy link
Member

sgiehl commented Dec 21, 2018

refs #3531

@mattab mattab added Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. and removed Needs Review PRs that need a code review labels Dec 31, 2018
@mattab mattab modified the milestones: 3.8.0, 3.9.0 Jan 6, 2019
@Findus23
Copy link
Member Author

Findus23 commented Feb 7, 2019

I'm afraid, we might have a problem:
https://developer.apple.com/documentation/safari_release_notes/safari_12_1_release_notes

Removed support for the expired Do Not Track standard to prevent potential use as a fingerprinting variable.

Can someone on iOS 12.2 or the next Mac OS beta please test, if this means that Apple stops sending the header at all (which would make their reason useless as they are now more identifiable than before) or if they just always send DNT:1 without a setting to change it.

The latter would mean quite some issues as we couldn't merge this PR.

@tsteur
Copy link
Member

tsteur commented Feb 7, 2019

Quickly checked browser stack but there iOS 12.1 is running

btw here a bit more on the background: https://www.gizmodo.com.au/2019/02/apple-is-removing-do-not-track-from-safari/

@Findus23
Copy link
Member Author

Findus23 commented Feb 9, 2019

I have now installed the 12.2 beta and indeed the option is gone.
But unlike I feared they are not sending the header by default, but just removed the header and the possibility to change it.
So this doesn't really have an influence on this PR.

@Findus23
Copy link
Member Author

In theory this PR is ready, but I totally fail to get the plugin event to work.

No matter what I change the handleDNTHeader() function is never executed. It seems like getPluginsLoadedAndActivated() doesn't include the plugin even though it is enabled.

Is there anything special a plugin has to do to work during tracking?

<?php
/**
 * Piwik - free/libre analytics platform
 *
 * @link http://piwik.org
 * @license http://www.gnu.org/licenses/gpl-3.0.html GPL v3 or later
 */

namespace Piwik\Plugins\IgnoreDNTByDefault;

use Piwik\Common;
use Piwik\Tracker\Request;

class IgnoreDNTByDefault extends \Piwik\Plugin
{
    public function registerEvents() {
        return array(
            'PrivacyManager.shouldIgnoreDnt' => 'handleDNTHeader'
        );
    }

    public function handleDNTHeader(&$shouldIgnore) {
        Common::printDebug($shouldIgnore);
        $shouldIgnore = $this->isUserAgentWithDoNotTrackAlwaysEnabled();
    }

    public function isUserAgentWithDoNotTrackAlwaysEnabled() {
        $request = new Request($_REQUEST);
        $userAgent = $request->getUserAgent();
        $browsersWithDnt = $this->getBrowsersWithDNTAlwaysEnabled();
        foreach ($browsersWithDnt as $userAgentBrowserFragment) {
            if (stripos($userAgent, $userAgentBrowserFragment) !== false) {
                return true;
            }
        }
        return false;
    }

    /**
     * Some browsers have DNT enabled by default. For those we will ignore DNT and always track those users.
     *
     * @return array
     */
    protected function getBrowsersWithDNTAlwaysEnabled() {
        return array(
            // IE
            'MSIE',
            'Trident',
            // Maxthon
            'Maxthon',

            // Epiphany - https://github.com/matomo-org/matomo/issues/8682
            'Epiphany',
        );
    }
}

@tsteur
Copy link
Member

tsteur commented Feb 10, 2019

You need to add a method

public function isTrackerPlugin(){
return true;
}

It can't detect it is a tracker plugin cause the event it is listening to doesn't start with Tracker or Tracking. During tracking for performance and security reasons etc we only load needed plugins.

@Findus23
Copy link
Member Author

Findus23 commented Feb 10, 2019

@tsteur Many thanks for the info, it works perfectly now.

The plugin can be found here: https://github.com/Findus23/plugin-IgnoreDNTEnabledByDefault

@Findus23 Findus23 added the Needs Review PRs that need a code review label Feb 10, 2019
@Findus23 Findus23 removed the Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. label Feb 10, 2019
@mattab mattab modified the milestones: 3.9.0, 3.10.0 Mar 18, 2019
@tsteur tsteur merged commit d6f49f2 into 3.x-dev Apr 11, 2019
@tsteur tsteur deleted the dont-ignore-DNT branch April 11, 2019 21:16
@Findus23
Copy link
Member Author

Totally missed this was merged.

Did someone have time to test the plugin (https://github.com/Findus23/plugin-IgnoreDNTEnabledByDefault)? @fdellwing ?

If so, I'd like to publish it before the release.

@Findus23 Findus23 restored the dont-ignore-DNT branch October 4, 2019 09:49
@sgiehl sgiehl deleted the dont-ignore-DNT branch February 10, 2020 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Platform For Matomo platform changes that aren't impacting any of our APIs but improve the core itself. Needs Review PRs that need a code review
Projects
None yet
5 participants