@utrautmann opened this Issue on June 9th 2022

The Region widget for State/Province shows a lot of unknown entries, although city entries could be localized.
The free database is used:
https://download.db-ip.com/free/dbip-city-lite-2022-06.mmdb.gz

The screenshots show the big difference between localized visitors for Regions and for Cities.
grafik
grafik

Current Behavior

I have checked this database resp. the CSV file from db-ip.com with the entries for State/Province.
The State/Province entry is sometimes for German regions in German language (Baden-Württemberg) or sometimes in English language (Lower Saxony, Hesse, Bavaria).

It seems to be that whenever the German provinces are in English language in the db-ip.com-Database, there will be an Region entry "Unknown" for the visitor in Matomo .

That might be a content related error at dp-ip.com, but my question is: how does Matomo work here?
What does Matomo need and are there any workarounds?
Needs Matomo the provinces in the language of the city?

  • Matomo Version: 4.10.1
@sgiehl commented on June 10th 2022 Member

Hi @utrautmann
That's kind of a problem with the DB-IP files. The free versions do only include the name of a region, but not the regions ISO code.
Matomo internally stores ISO codes only. If there is no ISO code provided by the geoip lookup, but a region name, we try to lookup the ISO code based on the country code and the region name. For that we are using a generated list of codes and names, that contains the names in it's countries language. See https://github.com/matomo-org/matomo/blob/4.x-dev/plugins/GeoIp2/data/isoRegionNames.php

So actually english names for german regions won't be mapped correctly.

@utrautmann commented on June 10th 2022

Hi @sgiehl ,
thank you for your answer. This helps me further understand how it works. I have now asked db-ip.com whether there are plans to provide the regions (provinces) in the national language again.

In Matomo, would it be possible that in the absence of an ISO code in the geolocation database, the region information would be used unchanged in Matomo, or would that have other implications?

@sgiehl commented on June 11th 2022 Member

@utrautmann We are storing ISO codes in the database and the database field won't be able to store more than 3 characters. So we can't simply store the name.
We could maybe try to additionally check for the english name of a region :thinking:

@utrautmann commented on June 13th 2022

@sgiehl
Here is the answer from db-ip.com.
It sounds like an approach that's the opposite of Matomo's.

"The IP to City Lite database uses English names when possible, or the local name if no translation is available."

@sgiehl commented on June 13th 2022 Member

Interesting. So you actually can't rely on what they would return, as you can't know what is translated or not 🤷
Still wondering why the can't simply include the ISO code 🙈
Anyway. The problem is, that we would need to fetch the english translations additionally, so we can compare that against the english names as well... And depending from where they get their translations they might even differ from the ones we might fetch 🤔

@utrautmann commented on June 16th 2022

@sgiehl:
I asked again. Here is the answer from db-ip.com:
"Unfortunately ISO codes and Geoname IDs are not present in the Lite databases and we have no plans to add these fields in the future".

Anyway. The problem is, that we would need to fetch the english translations additionally, so we can compare that against the english names as well... And depending from where they get their translations they might even differ from the ones we might fetch

First of all, I don't think that's a bad idea. Of course, it's possible that the English translation has an error, but in that case there's really not much you can do.

@sgiehl commented on June 20th 2022 Member

@justinvelluppillai We maybe should consider planning this one in soon. Using the lite version of db-ip.com, which we suggest by default, currently might report a lot unknown regions, due to an not map-able region name.
We currently have a command to update the list of regions, which we should run to update the official region names. In addition we should extend the command so it also tries to fetch the english name for each region. This could afterwards be used to check the region names coming from db-ip.com.

Another option might be to send an official request regarding including iso region codes into the lite database coming from Matomo. We might be a big referral for them, so maybe they would reconsider their plans 🤔

@dantefr commented on June 21st 2022

Hi,
The same problem exists with two French regions.
DB-IP -> MATOMO
Normandy -> Normandie
Brittany -> Bretagne

I'm not sure this is the right place, but there are too much dashes in the names of three French regions.
DB-IP -> MATOMO
Pays de la Loire -> Pays-de-la-Loire
Provence-Alpes-Côte d'Azur -> Provence-Alpes-Côte-d’Azur
Grand Est -> Grand-Est

I modified the "isoRegionNames.php" file, to have no more unknowns.

@MatomoForumNotifications commented on June 21st 2022

This issue has been mentioned on Matomo forums. There might be relevant details there:

https://forum.matomo.org/t/how-to-set-geolocation-to-wider-region-instead-of-city-suburb/46341/2

Powered by GitHub Issue Mirror