Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GeoLite2 Deprecated Coordinates in 2019 #12411

Closed
m-cameron opened this issue Jan 9, 2018 · 7 comments
Closed

GeoLite2 Deprecated Coordinates in 2019 #12411

m-cameron opened this issue Jan 9, 2018 · 7 comments
Labels
wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Milestone

Comments

@m-cameron
Copy link

m-cameron commented Jan 9, 2018

Maxmind has announced that they will remove coordinates from GeoLite2 in 2019.

https://dev.maxmind.com/geoip/geoip2/geolite2/

How can Piwik works without the coordinates? Must we subscribe to the commercial GeoIP2? Can we use IP2Location LITE DB5?

https://lite.ip2location.com/database/ip-country-region-city-latitude-longitude

@fdellwing
Copy link
Contributor

Why you need coordinates? City should be precise enough for 99% of the users that don't use a commercial solution anyway.

@Findus23
Copy link
Member

Findus23 commented Jan 9, 2018

I'm not sure if Piwik uses the coordinates to display the cities on the map.

@fdellwing
Copy link
Contributor

Well, the map works if you use other methods of geolocation instead of GeoLite DB. So there should be no problem with that?

@m-cameron
Copy link
Author

Need to check the codes if latitude and longitude are being used. Any developers here?

@mattab
Copy link
Member

mattab commented Apr 7, 2018

As maxmind itself says "Latitude and Longitude are often near the center of population. These values are not precise and should not be used to identify a particular address or household."

from internal discussions: we do not really need lat/long values for each user but the lat/long are currently used for the following:

One solution would be to have, in Matomo, a static mapping of cities -> lat,long and use this instead, so we know for sure lat/long are not more precise than the closest city. Then we could use this static mapping to fill in the lat/long info for each visit (even in the DB raw data) and keep our APIs and Maps working as before.

Whether to keep or delete the fields lat/long in the DB, because the values can be set manually via the Tracking API, many people use and rely on this valuable info (and may have full consent to track locations), so we can keep the fields in the tables.

@oschwald
Copy link

MaxMind is no longer discontinuing coordinates in GeoLite2:

After careful consideration, taking into account customer feedback, we have decided against removing latitude and longitude coordinates from the GeoLite2 databases. We are in the process of reviewing coordinates used in all of our GeoLite2 and GeoIP databases to ensure there is no risk of misuse.

@mattab
Copy link
Member

mattab commented May 2, 2018

Thanks for letting us know @oschwald - closing this as won't fix then.

@mattab mattab closed this as completed May 2, 2018
@mattab mattab added the wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. label May 2, 2018
@mattab mattab modified the milestones: 4.0.0, 3.5.0 May 2, 2018
mattab added a commit that referenced this issue May 2, 2018
sgiehl pushed a commit that referenced this issue May 3, 2018
sgiehl pushed a commit that referenced this issue May 3, 2018
mattab pushed a commit that referenced this issue May 4, 2018
* require geoip2/geoip2 composer package

* Determine region name based on Location Provider

* Adds empty GeoIp2 plugin

* move location_region column definition to GeoIp2 plugin
so it's get changed as soon as the plugin is activated

* Adds GeoIP2 location providers

* ignore GeoIP2 mmdb files

* Adds script to generate GeoIP2 test databases

* Adds Command to convert region codes from FIPS to ISO for old log table records

* Adds GeoIP2 AutoUpdater

* Use GeoIP2 in tests

* update test files

* code fixes

* adds tests

* rename old GeoIP providers to Legacy

* Let GeoIP autoupdater UI handle GeoIp2 as well

* convert region codes to ISO in API after switch to GeoIP2

* do not show GeoIP providers if GeoIP2 plugin is enabled an no GeoIP Legacy provider is still in use

* small fixes

* review changes

* Use correct region translations

* Show correct message if no database can be found

* if log tables have been converted, use archive date to check if region codes still need to be converted to iso

* fix tests

* Improves extracting GeoIP2 databases

* Adjust GeoLocation diagnostics

* readds old taiwan fixes

* Assume all third party location providers as 'recommended'

* Download database over HTTPS

* remove outdated comment (see #12411)

* Remove indication that Geoip2 may be slow, since we found it should be quite fast and this should not be an issue in theory

* skip detection if IP is empty & do not try convert IP to IPv4

* remove downloaded file if an error occurs while extracting

* command should be runnable multiple times

* use ISO codes for suggested region codes

* reload after wizard success

* Drop table if exists.

* Fix two translation keys.

* add special region handling for UK

* update system test

* update UI files

* submodule update

* update test files
InfinityVoid pushed a commit to InfinityVoid/matomo that referenced this issue Oct 11, 2018
* require geoip2/geoip2 composer package

* Determine region name based on Location Provider

* Adds empty GeoIp2 plugin

* move location_region column definition to GeoIp2 plugin
so it's get changed as soon as the plugin is activated

* Adds GeoIP2 location providers

* ignore GeoIP2 mmdb files

* Adds script to generate GeoIP2 test databases

* Adds Command to convert region codes from FIPS to ISO for old log table records

* Adds GeoIP2 AutoUpdater

* Use GeoIP2 in tests

* update test files

* code fixes

* adds tests

* rename old GeoIP providers to Legacy

* Let GeoIP autoupdater UI handle GeoIp2 as well

* convert region codes to ISO in API after switch to GeoIP2

* do not show GeoIP providers if GeoIP2 plugin is enabled an no GeoIP Legacy provider is still in use

* small fixes

* review changes

* Use correct region translations

* Show correct message if no database can be found

* if log tables have been converted, use archive date to check if region codes still need to be converted to iso

* fix tests

* Improves extracting GeoIP2 databases

* Adjust GeoLocation diagnostics

* readds old taiwan fixes

* Assume all third party location providers as 'recommended'

* Download database over HTTPS

* remove outdated comment (see matomo-org#12411)

* Remove indication that Geoip2 may be slow, since we found it should be quite fast and this should not be an issue in theory

* skip detection if IP is empty & do not try convert IP to IPv4

* remove downloaded file if an error occurs while extracting

* command should be runnable multiple times

* use ISO codes for suggested region codes

* reload after wizard success

* Drop table if exists.

* Fix two translation keys.

* add special region handling for UK

* update system test

* update UI files

* submodule update

* update test files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

5 participants