Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GeoIPAutoUpdater fails updating geoip2 database with .mmdb file ending #11049

Closed
slawa-dev opened this issue Dec 20, 2016 · 4 comments
Closed

GeoIPAutoUpdater fails updating geoip2 database with .mmdb file ending #11049

slawa-dev opened this issue Dec 20, 2016 · 4 comments
Assignees
@mattab
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Milestone
Backlog (Help wan...

Comments

@slawa-dev
Copy link

slawa-dev commented Dec 20, 2016 •
edited

Error: Encountered unsupported archive type 'mmdb.gz'

MaxMind geoip2 uses new format for goip database with .mmdb file ending.
e.g. GeoLite2-City.mmdb
http://dev.maxmind.com/geoip/geoip2/geolite2/

The download url for the file is:
http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz

cron/archive.php generates an error when auto updating.

INFO CoreAdminHome[2016-12-20 01:13:21] [d3190] Scheduler: executing task Piwik\Plugins\UserCountry\GeoIPAutoUpdater.update...
ERROR UserCountry[2016-12-20 01:13:21] [d3190] /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(489): Encountered unsupported archive type 'mmdb.gz'.
  #0 /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(471): Piwik\Plugins\UserCountry\GeoIPAutoUpdater::checkForSupportedArchiveType('mmdb.gz')
  #1 /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(134): Piwik\Plugins\UserCountry\GeoIPAutoUpdater::getGeoIPUrlExtension('http://geolite....')
  #2 /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(96): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
  #3 [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
  #4 /var/www/example.com/core/Scheduler/Scheduler.php(227): call_user_func(Array, NULL)
  #5 /var/www/example.com/core/Scheduler/Scheduler.php(124): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
  #6 /var/www/example.com/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
  #7 /var/www/example.com/core/CronArchive.php(536): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
  #8 /var/www/example.com/core/CronArchive.php(290): Piwik\CronArchive->runScheduledTasks()
  #9 /var/www/example.com/core/Access.php(469): Piwik\CronArchive->Piwik\{closure}()
  #10 /var/www/example.com/core/CronArchive.php(292): Piwik\Access::doAsSuperUser(Object(Closure))
  #11 /var/www/example.com/plugins/CoreAdminHome/API.php(179): Piwik\CronArchive->main()
  #12 [internal function]: Piwik\Plugins\CoreAdminHome\API->runCronArchiving()
  #13 /var/www/example.com/core/API/Proxy.php(219): call_user_func_array(Array, Array)
  #14 /var/www/example.com/core/API/Request.php(236): Piwik\API\Proxy->call('\\Piwik\\Plugins\\...', 'runCronArchivin...', Array)
  #15 /var/www/example.com/plugins/API/Controller.php(40): Piwik\API\Request->process()
  #16 [internal function]: Piwik\Plugins\API\Controller->index()
  #17 /var/www/example.com/core/FrontController.php(518): call_user_func_array(Array, Array)
  #18 /var/www/example.com/core/FrontController.php(122): Piwik\FrontController->doDispatch('API', false, Array)
  #19 /var/www/example.com/core/dispatch.php(34): Piwik\FrontController->dispatch()
  #20 /var/www/example.com/index.php(27): require_once('/var/www/exampl...')
  #21 /var/www/example.com/misc/cron/archive.php(70): require_once('/var/www/exampl...')
  #22 {main}
INFO CoreAdminHome[2016-12-20 01:13:21] [d3190] Scheduler: finished. Time elapsed: 0.013s
@slawa-dev slawa-dev mentioned this issue Dec 20, 2016
Closed
@slawa-dev slawa-dev changed the title GeoIPAutoUpdater fails with updating geoip2 database with .mmdb file ending GeoIPAutoUpdater fails updating geoip2 database with .mmdb file ending Dec 20, 2016
@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Dec 25, 2016
@mattab mattab mentioned this issue Dec 25, 2016
Merged
@mattab
Copy link
Member

mattab commented Dec 25, 2016

@S1awa thanks for the report. i've just added the extension to our white list in: https://github.com/piwik/piwik/pull/11081/files

Could you check if it works after updating the PR, since I haven't tested?

@mattab mattab added this to the 3.0.1 milestone Dec 25, 2016
@mattab mattab self-assigned this Dec 25, 2016
@slawa-dev
Copy link
Author

slawa-dev commented Dec 26, 2016 •
edited

@mattab

The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.

There are more checks for downloaded GeoIP Databases.

INFO UserCountry[2016-12-26 01:22:59] [5fd6a] GeoIPAutoUpdater: successfully downloaded 'http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz'
INFO UserCountry[2016-12-26 01:23:00] [5fd6a] GeoIPAutoUpdater: Encountered PHP error when testing newly downloaded GeoIP database: 8: Undefined offset: -16776960 on line 1705 of /home/www/analytics/libs/MaxMindGeoIP/geoip.inc.
ERROR UserCountry[2016-12-26 01:23:00] [5fd6a] /home/www/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(161): GeoIPAutoUpdater: failed to unzip '/home/www/analytics/misc/GeoIPCity.dat.mmdb.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.
  #0 /home/www/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(96): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
  #1 [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
  #2 /home/www/analytics/core/Scheduler/Scheduler.php(227): call_user_func(Array, NULL)
  #3 /home/www/analytics/core/Scheduler/Scheduler.php(124): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
  #4 /home/www/analytics/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
  #5 /home/www/analytics/core/CronArchive.php(536): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
  #6 /home/www/analytics/core/CronArchive.php(290): Piwik\CronArchive->runScheduledTasks()
  #7 /home/www/analytics/core/Access.php(469): Piwik\CronArchive->Piwik\{closure}()
  #8 /home/www/analytics/core/CronArchive.php(292): Piwik\Access::doAsSuperUser(Object(Closure))
  #9 /home/www/analytics/plugins/CoreAdminHome/API.php(179): Piwik\CronArchive->main()
  #10 [internal function]: Piwik\Plugins\CoreAdminHome\API->runCronArchiving()
  #11 /home/www/analytics/core/API/Proxy.php(219): call_user_func_array(Array, Array)
  #12 /home/www/analytics/core/API/Request.php(236): Piwik\API\Proxy->call('\\Piwik\\Plugins\\...', 'runCronArchivin...', Array)
  #13 /home/www/analytics/plugins/API/Controller.php(40): Piwik\API\Request->process()
  #14 [internal function]: Piwik\Plugins\API\Controller->index()
  #15 /home/www/analytics/core/FrontController.php(518): call_user_func_array(Array, Array)
  #16 /home/www/analytics/core/FrontController.php(122): Piwik\FrontController->doDispatch('API', false, Array)
  #17 /home/www/analytics/core/dispatch.php(34): Piwik\FrontController->dispatch()
  #18 /home/www/analytics/index.php(27): require_once('/home/www/anal...')
  #19 /home/www/analytics/misc/cron/archive.php(70): require_once('/home/www/anal...')
  #20 {main}
INFO CoreAdminHome[2016-12-26 01:23:00] [5fd6a] Scheduler: finished. Time elapsed: 0.803s

@slawa-dev
Copy link
Author

slawa-dev commented Jan 25, 2017 •
edited

@mattab
GeoIP2 Plugin is buggy atm.
So the fix might work when the plugin is fixed.

#11157

@mxmilkiib mxmilkiib mentioned this issue May 14, 2018
Closed
@sgiehl
Copy link
Member

sgiehl commented May 14, 2018

Matomo 3.5.0 now includes a GeoIp2 plugin. This fully support the new mmdb format.

@sgiehl sgiehl closed this as completed May 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mattab mattab

Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@mattab @sgiehl @slawa-dev