@dev-101 opened this Issue on August 29th 2020

Hi, it would be very useful if we could tell Matomo to either:

**1. only track visits from specific target countries

  1. or completely exclude visitors from specific countries**

By exclude or track, I mean log into database or completely ignore / drop them. This would save so much resources, streamline our analytics work and keep database resources low.

Yes, I know we have post-filters, but that's not it. It is not good enough.

Also, yeah, we can use 3rd party scripts, but problem with custom country blockers is that it adds additional layer of complexity on top of Matomo, we have to maintain a script that depends on 3rd party updates etc. and Matomo already has all that built-in. Once it determines a country from a visitor's IP, it can simply decide whether it should store it or drop it.

Is there a way to actually build a simple plugin that can do this for us? It should be a simple hard-coded array of countries, I don't mind. It does not have to work in real-time, say, a cron-based script that can clean-up all entries in database every 24 hours that match certain criteria (e.g. visitor's country field). Similar to what we have in GDPR tools now.

Thanks!

@Findus23 commented on August 29th 2020 Member

Hi,

I think this might be easily implemented as a plugin that during tracking checks if the detected country match your list and if so, simply drop the request (similar to DNT).

In theory listening to this event
https://github.com/matomo-org/matomo/blob/17ec9004f99b6f7092740558bca71cca6c1ac23c/core/Tracker/VisitExcluded.php#L97
should be enough, but I am not sure if at this point Geolocation has already run. If so, it should be pretty easy to implement.

@dev-101 commented on August 29th 2020

Hi, thanks, I do wish to go that route at first, this is just a suggestion, as I think this actually can be a very useful feature.
I don't know where to start with the plugin, though, I have to really dig into core and see what methods are used to for e.g. remove visits on GDPR page.

add: I see your edited answer. I will try to test it later today. Thanks!

@Findus23 commented on August 29th 2020 Member

It seems like my simple idea using Tracker.isExcludedVisit doesn't work as at that point one only has Request which holds the parameters of the http request. I still think that excluding during the tracker is a better idea than retroactively removing them from the database as that means invalidating and reprocessing a lot of reports.

@sgiehl commented on August 29th 2020 Member

@dev-101 actually that could work using Tracker.isExcluded event. Adding something like this into a config.php in config directory should work:

<?php

return [
    'observers.global' => \DI\add([[
        'Tracker.isExcludedVisit',
            function(&$excluded, \Piwik\Tracker\Request $request) {

                $provider = \Piwik\Plugins\UserCountry\LocationProvider::getProviderById(\Piwik\Common::getCurrentLocationProviderId());

                $location = $provider->getLocation(['ip' => $request->getIp()]);

                if (strtolower($location[\Piwik\Plugins\UserCountry\LocationProvider::COUNTRY_CODE_KEY]) !== 'de') {
                    $excluded = true;
                }

            }
    ]])
];

That should exclude all visits not coming from Germany.

Note: This might not be the optimal solution, as would do the geo location twice per request.

@dev-101 commented on August 29th 2020

Thank you, it looks promising, although putting this into config.ini.php will probably get overwritten during the next update (I assume). Wonder if that code can be placed inside a plugin?

I have added the code, but for now I am just simulating it. Thanks!

@dev-101 commented on August 29th 2020

Unfortunately, it's not working:

PHP Fatal error: Uncaught Piwik\Ini\IniReadingException: Unable to read INI file {.../config/config.ini.php}: Syntax error in INI configuration: syntax error, unexpected $end, expecting ']' in Unknown on line 4

It seems that we must not put PHP code inside ini file, as it confuses Piwik/Matomo.


update: I tried putting this code inside return array of global.php (seems "logical"), but can't say it is working.

@Findus23 commented on August 30th 2020 Member

The hint by @sgiehl helped me create a plugin that at least in my limited test seems to work fine:
https://github.com/Findus23/plugin-ExcludeCountries
you can download the zip from https://github.com/Findus23/plugin-ExcludeCountries/archive/main.zip, rename the directory to ExcludeCountries, move it to plugins/, enable it in the plugin view, remove the part from the global.php and check if it is working with https://developer.matomo.org/api-reference/tracking-api#debugging-the-tracker

DEBUG ExcludeCountries[2020-08-30 08:55:31 UTC] [519fe] request excluded by ExcludeCountries plugin (at)
DEBUG CoreHome[2020-08-30 08:55:31 UTC] [519fe] Visitor excluded.
@dev-101 commented on August 30th 2020

Thank you, Findus23!
I have installed it and configured, now have to wait & see. Many thanks!

@dev-101 commented on August 30th 2020

minor issue:

PHP Notice: Undefined index: languageCode in /ExcludeCountries/ExcludeCountries.php on line 69

But, so far, tracking of included countries seems to be working fine.

@Findus23 commented on August 30th 2020 Member

@dev-101 Not sure why exactly this happens, but I now exclude invalid settings.

@dev-101 commented on August 30th 2020

According to time stamps, it was at the time I installed and configured plugin, there were like 20-30 lines of above notice in the log. Afterwards nothing, so it works ok. I "fixed" it with @

@dev-101 commented on August 31st 2020

I will close this issue now.
Any bug report re plugin will be posted in its own repo.

Thanks everyone 👍

@tsteur commented on August 31st 2020 Member

@dev-101 I'll reopen the issue for now just in case anyone wants to develop something like this as a plugin

@dev-101 commented on August 31st 2020

Ok :)

btw. Findus23's plugin works great so far... it's a done deal imho. It is fully configurable in the General Settings page, at least, it satisfies both points 1. and 2. from the opening post.

@Findus23 commented on September 1st 2020 Member
@dev-101 commented on December 4th 2020

Hi @Findus23 could you please bump plugin compatibility with Matomo 4? Thanks!

edit: I see Matomo pulled updated version automatically after update, I received a warning before upgrade about compatibility and that it will be disabled after upgrade. Sorry.

@Findus23 commented on December 4th 2020 Member

Hi @dev-101,

No problem. I wasn't sure myself if I didn't just forget the plugin.

@Findus23 commented on December 31st 2020 Member

@dev-101 Just for your information: There is an official plugin currently in development that supports everything my plugin can do and even more:

https://github.com/matomo-org/plugin-TrackingSpamPrevention

@dev-101 commented on December 31st 2020

Thanks, this is great! Happy NY :)

@dev-101 commented on July 11th 2021

Just a heads-up if someone is using this plugin (version 0.2.0), it caused tracking issues on my website out-of-the-blue -- last 4 days tracking stopped and here's the report:

Last 4 days one of my Matomo installation (shared hosting, cPanel, Linux, PHP 7.3.8) stopped tracking visitors. GA works fine.

Here's what I found in system reports:

ERROR The configured geolocation provider DBIP / GeoIP 2 (Php) is broken. Please fix the provider or configure another one to get geolocation working again.

matomo-sys-check

matomo-js-500-error

```[11-Jul-2021 15:33:58 UTC] PHP Fatal error: Uncaught Error: Call to a member function getLocation() on null in /matomo/plugins/ExcludeCountries/ExcludeCountries.php:73
Stack trace:
#0 [internal function]: Piwik\Plugins\ExcludeCountries\ExcludeCountries->checkExcludedCountry(false, Object(Piwik\Tracker\Request))
#1 /matomo/core/EventDispatcher.php(141): call_user_func_array(Array, Array)
#2 /matomo/core/Piwik.php(809): Piwik\EventDispatcher->postEvent('Tracker.isExclu...', Array, false, Array)
#3 /matomo/core/Tracker/VisitExcluded.php(97): Piwik\Piwik::postEvent('Tracker.isExclu...', Array)
#4 /matomo/plugins/CoreHome/Tracker/VisitRequestProcessor.php(100): Piwik\Tracker\VisitExcluded->isExcluded()
#5 /matomo/core/Tracker/Visit.php(164): Piwik\Plugins\CoreHome\Tracker\VisitRequestProcessor->processRequestParams(Object(Piwik\Tracker\Visit\VisitProperties), Object(Piwik\Tracker\Request))
#6 in /matomo/plugins/ExcludeCountries/ExcludeCountries.php on line 73



I **haven't changed** anything in the past few months besides updating Matomo.

Contrary to what above reports might indicate, the issue is within this plugin, simply deactivating it resolved tracking and other problems right away.

I know there was a work on a new plugin, but I didn't switch to it. Maybe that was the problem...

edit: switched to newer plugin, problem fixed.
This Issue was closed on September 1st 2020
Powered by GitHub Issue Mirror