Good progress :+1: from the check list it looks like all the work is done, as we don't really need "trigger update in browser", let's skip this one. Is the work complete and ready to be reviewed & merged?
Almost. I will update the list of tasks. But I would disagree with "update in browser". As we don't do the update within the normal update progress we make it impossible to migrate for users without access to the console. Which applies for most users on a shared hosting,
I've now finished updating the FIPS => ISO mapping manually. All old codes should now be in the list.
The list of region names currently included may lack some names, as it is currently generated using data from iso-codes. And unfortunately their list seems not to be very up to date. (But I've already added some missing entries, where I've seen missing data while testing)
If we want to have a complete list of names, we might need to purchase the official ISO standard.
I will now have another look at all changes and afterwards it should be good for a first review
@sgiehl Please find my first review and some questions below:
ALTER TABLE `piwik_log_visit` MODIFY COLUMN `location_region` char(3) DEFAULT NULL; ALTER TABLE `piwik_log_conversion` MODIFY COLUMN `location_region` char(3) DEFAULT NULL;
-> as we can't introduce schema changes on
log_* table within a major release cycle, should we move the schema changes to Matomo 4.0.0 milestone? Or what are our options ie. is it totally broken if we keep 2 chars region codes?
Geolocation: remove the GeoIP Legacy Support
Schema change is required as GeoIP2 returns ISO region codes and those can have up to three characters. So if we want to switch to ISO codes we need to change the schema. Converting the ISO data returned by GeoIP2 to the old FIPS codes does not seem to be a good option. I would estimate that it takes around 3 days to create a proper mapping list. Also we would loose some information as not all ISO codes can be mapped to FIPS. So imho we should move GeoIP2 to Matomo 4, if a schema change can't be done before.
GeoIP 2 ISP/ORG databases are still supported, but they have been merged into one. See https://www.maxmind.com/en/geoip2-isp-database
We should not (yet) remove GeoIP legacy support. Maxmind will still provide updates for the paid databases. Only updates for GeoIP Lite are discontinued.
Let's update the FAQs as soon as we have a plan when this will be merged.
Regarding performance: Would need to set up some tests to check how fast the implementations are.
As discussed internally here is a possible way to solve the upgrade path & our inability to introduce a schema change to existing users:
Shall we proceed with this solution? Looking forward to merge the work and allowing users to leverage much increased accuracy of Geoip2!
Just realized switching to FIPS region codes will break the visitor map. The map currently handles FIPS codes, and I don't think that can be easily changed...