@dev-101 opened this Issue on August 29th 2020

Hi, it would be very useful if we could tell Matomo to either:

**1. only track visits from specific target countries

  1. or completely exclude visitors from specific countries**

By exclude or track, I mean log into database or completely ignore / drop them. This would save so much resources, streamline our analytics work and keep database resources low.

Yes, I know we have post-filters, but that's not it. It is not good enough.

Also, yeah, we can use 3rd party scripts, but problem with custom country blockers is that it adds additional layer of complexity on top of Matomo, we have to maintain a script that depends on 3rd party updates etc. and Matomo already has all that built-in. Once it determines a country from a visitor's IP, it can simply decide whether it should store it or drop it.

Is there a way to actually build a simple plugin that can do this for us? It should be a simple hard-coded array of countries, I don't mind. It does not have to work in real-time, say, a cron-based script that can clean-up all entries in database every 24 hours that match certain criteria (e.g. visitor's country field). Similar to what we have in GDPR tools now.

Thanks!

@Findus23 commented on August 29th 2020 Member

Hi,

I think this might be easily implemented as a plugin that during tracking checks if the detected country match your list and if so, simply drop the request (similar to DNT).

In theory listening to this event
https://github.com/matomo-org/matomo/blob/17ec9004f99b6f7092740558bca71cca6c1ac23c/core/Tracker/VisitExcluded.php#L97
should be enough, but I am not sure if at this point Geolocation has already run. If so, it should be pretty easy to implement.

@dev-101 commented on August 29th 2020

Hi, thanks, I do wish to go that route at first, this is just a suggestion, as I think this actually can be a very useful feature.
I don't know where to start with the plugin, though, I have to really dig into core and see what methods are used to for e.g. remove visits on GDPR page.

add: I see your edited answer. I will try to test it later today. Thanks!

@Findus23 commented on August 29th 2020 Member

It seems like my simple idea using Tracker.isExcludedVisit doesn't work as at that point one only has Request which holds the parameters of the http request. I still think that excluding during the tracker is a better idea than retroactively removing them from the database as that means invalidating and reprocessing a lot of reports.

@sgiehl commented on August 29th 2020 Member

@dev-101 actually that could work using Tracker.isExcluded event. Adding something like this into a config.php in config directory should work:

<?php

return [
    'observers.global' => \DI\add([[
        'Tracker.isExcludedVisit',
            function(&$excluded, \Piwik\Tracker\Request $request) {

                $provider = \Piwik\Plugins\UserCountry\LocationProvider::getProviderById(\Piwik\Common::getCurrentLocationProviderId());

                $location = $provider->getLocation(['ip' => $request->getIp()]);

                if (strtolower($location[\Piwik\Plugins\UserCountry\LocationProvider::COUNTRY_CODE_KEY]) !== 'de') {
                    $excluded = true;
                }

            }
    ]])
];

That should exclude all visits not coming from Germany.

Note: This might not be the optimal solution, as would do the geo location twice per request.

@dev-101 commented on August 29th 2020

Thank you, it looks promising, although putting this into config.ini.php will probably get overwritten during the next update (I assume). Wonder if that code can be placed inside a plugin?

I have added the code, but for now I am just simulating it. Thanks!

@dev-101 commented on August 29th 2020

Unfortunately, it's not working:

PHP Fatal error: Uncaught Piwik\Ini\IniReadingException: Unable to read INI file {.../config/config.ini.php}: Syntax error in INI configuration: syntax error, unexpected $end, expecting ']' in Unknown on line 4

It seems that we must not put PHP code inside ini file, as it confuses Piwik/Matomo.


update: I tried putting this code inside return array of global.php (seems "logical"), but can't say it is working.

@Findus23 commented on August 30th 2020 Member

The hint by @sgiehl helped me create a plugin that at least in my limited test seems to work fine:
https://github.com/Findus23/plugin-ExcludeCountries
you can download the zip from https://github.com/Findus23/plugin-ExcludeCountries/archive/main.zip, rename the directory to ExcludeCountries, move it to plugins/, enable it in the plugin view, remove the part from the global.php and check if it is working with https://developer.matomo.org/api-reference/tracking-api#debugging-the-tracker

DEBUG ExcludeCountries[2020-08-30 08:55:31 UTC] [519fe] request excluded by ExcludeCountries plugin (at)
DEBUG CoreHome[2020-08-30 08:55:31 UTC] [519fe] Visitor excluded.
@dev-101 commented on August 30th 2020

Thank you, Findus23!
I have installed it and configured, now have to wait & see. Many thanks!

@dev-101 commented on August 30th 2020

minor issue:

PHP Notice: Undefined index: languageCode in /ExcludeCountries/ExcludeCountries.php on line 69

But, so far, tracking of included countries seems to be working fine.

@Findus23 commented on August 30th 2020 Member

@dev-101 Not sure why exactly this happens, but I now exclude invalid settings.

@dev-101 commented on August 30th 2020

According to time stamps, it was at the time I installed and configured plugin, there were like 20-30 lines of above notice in the log. Afterwards nothing, so it works ok. I "fixed" it with @

@dev-101 commented on August 31st 2020

I will close this issue now.
Any bug report re plugin will be posted in its own repo.

Thanks everyone 👍

@tsteur commented on August 31st 2020 Member

@dev-101 I'll reopen the issue for now just in case anyone wants to develop something like this as a plugin

@dev-101 commented on August 31st 2020

Ok :)

btw. Findus23's plugin works great so far... it's a done deal imho. It is fully configurable in the General Settings page, at least, it satisfies both points 1. and 2. from the opening post.

@Findus23 commented on September 1st 2020 Member
This Issue was closed on September 1st 2020
Powered by GitHub Issue Mirror