@s1awa opened this Issue on December 20th 2016

Error: Encountered unsupported archive type 'mmdb.gz'

MaxMind geoip2 uses new format for goip database with .mmdb file ending.
e.g. GeoLite2-City.mmdb
http://dev.maxmind.com/geoip/geoip2/geolite2/

The download url for the file is:
http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz

cron/archive.php generates an error when auto updating.

INFO CoreAdminHome[2016-12-20 01:13:21] [d3190] Scheduler: executing task Piwik\Plugins\UserCountry\GeoIPAutoUpdater.update...
ERROR UserCountry[2016-12-20 01:13:21] [d3190] /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(489): Encountered unsupported archive type 'mmdb.gz'.
  <a href='/0'>#0</a> /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(471): Piwik\Plugins\UserCountry\GeoIPAutoUpdater::checkForSupportedArchiveType('mmdb.gz')
  <a href='/1'>#1</a> /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(134): Piwik\Plugins\UserCountry\GeoIPAutoUpdater::getGeoIPUrlExtension('http://geolite....')
  <a href='/2'>#2</a> /var/www/example.com/plugins/UserCountry/GeoIPAutoUpdater.php(96): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
  <a href='/3'>#3</a> [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
  <a href='/4'>#4</a> /var/www/example.com/core/Scheduler/Scheduler.php(227): call_user_func(Array, NULL)
  <a href='/5'>#5</a> /var/www/example.com/core/Scheduler/Scheduler.php(124): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
  <a href='/6'>#6</a> /var/www/example.com/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
  <a href='/7'>#7</a> /var/www/example.com/core/CronArchive.php(536): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
  <a href='/8'>#8</a> /var/www/example.com/core/CronArchive.php(290): Piwik\CronArchive->runScheduledTasks()
  <a href='/9'>#9</a> /var/www/example.com/core/Access.php(469): Piwik\CronArchive->Piwik\{closure}()
  <a href='/10'>#10</a> /var/www/example.com/core/CronArchive.php(292): Piwik\Access::doAsSuperUser(Object(Closure))
  <a href='/11'>#11</a> /var/www/example.com/plugins/CoreAdminHome/API.php(179): Piwik\CronArchive->main()
  <a href='/12'>#12</a> [internal function]: Piwik\Plugins\CoreAdminHome\API->runCronArchiving()
  <a href='/13'>#13</a> /var/www/example.com/core/API/Proxy.php(219): call_user_func_array(Array, Array)
  <a href='/14'>#14</a> /var/www/example.com/core/API/Request.php(236): Piwik\API\Proxy->call('\\Piwik\\Plugins\\...', 'runCronArchivin...', Array)
  <a href='/15'>#15</a> /var/www/example.com/plugins/API/Controller.php(40): Piwik\API\Request->process()
  <a href='/16'>#16</a> [internal function]: Piwik\Plugins\API\Controller->index()
  <a href='/17'>#17</a> /var/www/example.com/core/FrontController.php(518): call_user_func_array(Array, Array)
  <a href='/18'>#18</a> /var/www/example.com/core/FrontController.php(122): Piwik\FrontController->doDispatch('API', false, Array)
  <a href='/19'>#19</a> /var/www/example.com/core/dispatch.php(34): Piwik\FrontController->dispatch()
  <a href='/20'>#20</a> /var/www/example.com/index.php(27): require_once('/var/www/exampl...')
  <a href='/21'>#21</a> /var/www/example.com/misc/cron/archive.php(70): require_once('/var/www/exampl...')
  <a href='/22'>#22</a> {main}
INFO CoreAdminHome[2016-12-20 01:13:21] [d3190] Scheduler: finished. Time elapsed: 0.013s
@mattab commented on December 25th 2016 Member

@s1awa thanks for the report. i've just added the extension to our white list in: https://github.com/piwik/piwik/pull/11081/files

Could you check if it works after updating the PR, since I haven't tested?

@s1awa commented on December 26th 2016

@mattab

The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.

There are more checks for downloaded GeoIP Databases.

INFO UserCountry[2016-12-26 01:22:59] [5fd6a] GeoIPAutoUpdater: successfully downloaded 'http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz'
INFO UserCountry[2016-12-26 01:23:00] [5fd6a] GeoIPAutoUpdater: Encountered PHP error when testing newly downloaded GeoIP database: 8: Undefined offset: -16776960 on line 1705 of /home/www/analytics/libs/MaxMindGeoIP/geoip.inc.
ERROR UserCountry[2016-12-26 01:23:00] [5fd6a] /home/www/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(161): GeoIPAutoUpdater: failed to unzip '/home/www/analytics/misc/GeoIPCity.dat.mmdb.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.mmdb.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.
  <a href='/0'>#0</a> /home/www/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(96): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
  <a href='/1'>#1</a> [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
  <a href='/2'>#2</a> /home/www/analytics/core/Scheduler/Scheduler.php(227): call_user_func(Array, NULL)
  <a href='/3'>#3</a> /home/www/analytics/core/Scheduler/Scheduler.php(124): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
  <a href='/4'>#4</a> /home/www/analytics/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
  <a href='/5'>#5</a> /home/www/analytics/core/CronArchive.php(536): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
  <a href='/6'>#6</a> /home/www/analytics/core/CronArchive.php(290): Piwik\CronArchive->runScheduledTasks()
  <a href='/7'>#7</a> /home/www/analytics/core/Access.php(469): Piwik\CronArchive->Piwik\{closure}()
  <a href='/8'>#8</a> /home/www/analytics/core/CronArchive.php(292): Piwik\Access::doAsSuperUser(Object(Closure))
  <a href='/9'>#9</a> /home/www/analytics/plugins/CoreAdminHome/API.php(179): Piwik\CronArchive->main()
  <a href='/10'>#10</a> [internal function]: Piwik\Plugins\CoreAdminHome\API->runCronArchiving()
  <a href='/11'>#11</a> /home/www/analytics/core/API/Proxy.php(219): call_user_func_array(Array, Array)
  <a href='/12'>#12</a> /home/www/analytics/core/API/Request.php(236): Piwik\API\Proxy->call('\\Piwik\\Plugins\\...', 'runCronArchivin...', Array)
  <a href='/13'>#13</a> /home/www/analytics/plugins/API/Controller.php(40): Piwik\API\Request->process()
  <a href='/14'>#14</a> [internal function]: Piwik\Plugins\API\Controller->index()
  <a href='/15'>#15</a> /home/www/analytics/core/FrontController.php(518): call_user_func_array(Array, Array)
  <a href='/16'>#16</a> /home/www/analytics/core/FrontController.php(122): Piwik\FrontController->doDispatch('API', false, Array)
  <a href='/17'>#17</a> /home/www/analytics/core/dispatch.php(34): Piwik\FrontController->dispatch()
  <a href='/18'>#18</a> /home/www/analytics/index.php(27): require_once('/home/www/anal...')
  <a href='/19'>#19</a> /home/www/analytics/misc/cron/archive.php(70): require_once('/home/www/anal...')
  <a href='/20'>#20</a> {main}
INFO CoreAdminHome[2016-12-26 01:23:00] [5fd6a] Scheduler: finished. Time elapsed: 0.803s
@s1awa commented on January 25th 2017

@mattab
GeoIP2 Plugin is buggy atm.
So the fix might work when the plugin is fixed.

https://github.com/matomo-org/matomo/issues/11157

@sgiehl commented on May 14th 2018 Member

Matomo 3.5.0 now includes a GeoIp2 plugin. This fully support the new mmdb format.

This Issue was closed on May 14th 2018
Powered by GitHub Issue Mirror