@mattab opened this Issue on January 18th 2016 Member

GeoIPAutoUpdater: failed to unzip '/var/www/stat/misc/GeoIPCity.dat.dat.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.

As reported in http://forum.piwik.org/t/geoipautoupdater-error/16143

@mattab commented on January 18th 2016 Member

Notes

  • This is reproducible on demo and demo2. I didn't notice but we did get email notification errors from the cron
  • that there is twice .dat in the filename, which maybe indicates an error (or maybe not?) failed to unzip '/var/www/stat/misc/GeoIPCity.dat.dat.gz.

Full error trace is:

ERROR [2016-01-06 00:56:32] /home/piwik-demo/www/demo.piwik.org/plugins/UserCountry/GeoIPAutoUpdater.php(161): GeoIPAutoUpdater: failed to unzip '/home/piwik-demo/www/demo.piwik.org/misc/GeoIPCity.dat.dat.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.
<a href='/0'>#0</a> /home/piwik-demo/www/demo.piwik.org/plugins/UserCountry/GeoIPAutoUpdater.php(96): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
<a href='/1'>#1</a> [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
<a href='/2'>#2</a> /home/piwik-demo/www/demo.piwik.org/core/Scheduler/Scheduler.php(227): call_user_func(Array, NULL)
<a href='/3'>#3</a> /home/piwik-demo/www/demo.piwik.org/core/Scheduler/Scheduler.php(124): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
<a href='/4'>#4</a> /home/piwik-demo/www/demo.piwik.org/plugins/CoreAdminHome/API.php(55): Piwik\Scheduler\Scheduler->run()
<a href='/5'>#5</a> /home/piwik-demo/www/demo.piwik.org/core/CronArchive.php(529): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
<a href='/6'>#6</a> /home/piwik-demo/www/demo.piwik.org/core/CronArchive.php(283): Piwik\CronArchive->runScheduledTasks()
<a href='/7'>#7</a> /home/piwik-demo/www/demo.piwik.org/core/Access.php(466): Piwik\CronArchive->Piwik\{closure}()
<a href='/8'>#8</a> /home/piwik-demo/www/demo.piwik.org/core/CronArchive.php(285): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/9'>#9</a> /home/piwik-demo/www/demo.piwik.org/plugins/CoreConsole/Commands/CoreArchiver.php(27): Piwik\CronArchive->main()
<a href='/10'>#10</a> /home/piwik-demo/www/demo.piwik.org/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/11'>#11</a> /home/piwik-demo/www/demo.piwik.org/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/12'>#12</a> /home/piwik-demo/www/demo.piwik.org/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/13'>#13</a> [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/14'>#14</a> /home/piwik-demo/www/demo.piwik.org/core/Console.php(79): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/15'>#15</a> /home/piwik-demo/www/demo.piwik.org/core/Access.php(466): Piwik\Console->Piwik\{closure}()
<a href='/16'>#16</a> /home/piwik-demo/www/demo.piwik.org/core/Console.php(80): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/17'>#17</a> /home/piwik-demo/www/demo.piwik.org/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/18'>#18</a> /home/piwik-demo/www/demo.piwik.org/misc/cron/archive.php(57): Symfony\Component\Console\Application->run()
<a href='/19'>#19</a> {main}
@mattab commented on January 18th 2016 Member

Adding to 2.16.0 - @tsteur @sgiehl maybe you could take a look at this one? It started occuring in November and seems to be experienced by all users.

@tsteur commented on January 18th 2016 Member

FYI: geoip.loc_db_url in option table in DB says http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz on demo. Had a quick look but code is almost impossible to understand so needs proper investigating or refactoring.

@hpvd commented on January 18th 2016

in our installation (2.16b3) in backend there is no error message visible.
Looks "working ok"
=> is it ok in this installation (and does not appear everywhere)?
=> or should one add a error massage to report if there is a problem?
please see attachement

2016-01-18_23h06_34

@clauso commented on February 3rd 2016

I have the same error today (I think I have seen it in the past):

ERROR [2016-02-03 00:05:11] /var/www/piwik/plugins/UserCountry/GeoIPAutoUpdater.php(161): GeoIPAutoUpdater: failed to unzip '/var/www/piwik/misc/GeoIPCity.dat.dat.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.

We are running piwik 2.15.0

I have this md5sum for the downloaded file:

$> md5sum /var/www/piwik/misc/GeoLiteCity.dat
97752fa1e1407d9c2c48008a4b9428c1  /var/www/piwik/misc/GeoLiteCity.dat
@nd2s commented on March 5th 2016

Fails here as well. v2.16.0 on FreeBSD.

@clauso commented on July 13th 2016

Still get those mails about failures so I tried to look into it. I found out that I can trigger an update run manually by calling

php /var/www/piwik/console scheduled-tasks:run --force

A maybe interesting thing is that here I get an additional error that was not in the mails, see line 2:

INFO [2016-07-13 04:59:50] GeoIPAutoUpdater: successfully downloaded 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz'
INFO [2016-07-13 04:59:50] GeoIPAutoUpdater: Encountered PHP error when testing newly downloaded GeoIP database: 2: include(): Failed opening '/var/www/piwik/tmp/cache/tracker/piwikcache_Translations-en-e21bf04aeec884480f77854677fd2bf5fbc090f3.php' for inclusion (include_path='/var/www/piwik/vendor/pear/pear_exception:/var/www/piwik/vendor/pear/console_getopt:/var/www/piwik/vendor/pear/pear-core-minimal/src:/var/www/piwik/vendor/pear/archive_tar:.:/usr/share/php') on line 112 of /var/www/piwik/vendor/doctrine/cache/lib/Doctrine/Common/Cache/PhpFileCache.php.
ERROR [2016-07-13 04:59:50] /var/www/piwik/plugins/UserCountry/GeoIPAutoUpdater.php(161): GeoIPAutoUpdater: failed to unzip '/var/www/piwik/misc/GeoIPCity.dat.dat.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.

I currently do not have the piwik knowledge and time to investigate further but maybe this give the right person a hint?

@jonroberts-anr commented on May 14th 2017

Please turn JavaScript on and reload the page.

Checking your browser before accessing maxmind.com.

This process is automatic. Your browser will redirect to your requested content shortly.

Please allow up to 5 seconds…

DDoS protection by Cloudflare
Ray ID: 35e99544e3061049
@robertoschwald commented on March 28th 2018

Any news on this? Facing this problem since a long time

@JonTheWong commented on January 14th 2019

I've been recently getting the same error.
My current error output is for GeoLite2 but I've tried legacy with no success. The URL is giving me a 404 on legacy. Geo2 doesn't work because it doesn't contain a .dat file.

System Check
"Huzzah! There are no problems with your Matomo setup. Give yourself a pat on the back."

Matomo 3.7.0 Current.

ERROR [2019-01-13 11:49:21] 6701  /home/user/public_html/analytics.example.com/plugins/UserCountry/GeoIPAutoUpdater.php(167): GeoIPAutoUpdater: failed to unzip '/home/user/public_html/analytics.example.com/misc/GeoIPCity.dat.tar.gz' after downloading 'https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz': Cannot find GeoIPCity.dat file in tar archive '/home/user/public_html/analytics.example.com/misc/GeoIPCity.dat.tar.gz'!
<a href='/0'>#0</a> /home/user/public_html/analytics.example.com/plugins/UserCountry/GeoIPAutoUpdater.php(102): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'https://geolite...')
<a href='/1'>#1</a> [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
<a href='/2'>#2</a> /home/user/public_html/analytics.example.com/core/Scheduler/Scheduler.php(246): call_user_func(Array, NULL)
<a href='/3'>#3</a> /home/user/public_html/analytics.example.com/core/Scheduler/Scheduler.php(136): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
<a href='/4'>#4</a> /home/user/public_html/analytics.example.com/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
<a href='/5'>#5</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(612): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
<a href='/6'>#6</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(317): Piwik\CronArchive->runScheduledTasks()
<a href='/7'>#7</a> /home/user/public_html/analytics.example.com/core/Access.php(626): Piwik\CronArchive->Piwik\{closure}()
<a href='/8'>#8</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(319): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/9'>#9</a> /home/user/public_html/analytics.example.com/plugins/CoreConsole/Commands/CoreArchiver.php(27): Piwik\CronArchive->main()
<a href='/10'>#10</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/11'>#11</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/12'>#12</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/13'>#13</a> [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/14'>#14</a> /home/user/public_html/analytics.example.com/core/Console.php(90): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/15'>#15</a> /home/user/public_html/analytics.example.com/core/Access.php(626): Piwik\Console->Piwik\{closure}()
<a href='/16'>#16</a> /home/user/public_html/analytics.example.com/core/Console.php(91): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/17'>#17</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/18'>#18</a> /home/user/public_html/analytics.example.com/console(27): Symfony\Component\Console\Application->run()
<a href='/19'>#19</a> {main}

Found the original log.

ERROR [2019-01-09 11:45:50] 12232  /home/user/public_html/analytics.example.com/plugins/UserCountry/GeoIPAutoUpdater.php(167): GeoIPAutoUpdater: failed to unzip '/home/user/public_html/analytics.example.com/misc/GeoIPCity.dat.dat.gz' after downloading 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz': The downloaded file is not a valid GeoIP database. Please re-check the URL or download the file manually.
<a href='/0'>#0</a> /home/user/public_html/analytics.example.com/plugins/UserCountry/GeoIPAutoUpdater.php(102): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'http://geolite....')
<a href='/1'>#1</a> [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
<a href='/2'>#2</a> /home/user/public_html/analytics.example.com/core/Scheduler/Scheduler.php(246): call_user_func(Array, NULL)
<a href='/3'>#3</a> /home/user/public_html/analytics.example.com/core/Scheduler/Scheduler.php(136): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
<a href='/4'>#4</a> /home/user/public_html/analytics.example.com/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
<a href='/5'>#5</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(612): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
<a href='/6'>#6</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(317): Piwik\CronArchive->runScheduledTasks()
<a href='/7'>#7</a> /home/user/public_html/analytics.example.com/core/Access.php(626): Piwik\CronArchive->Piwik\{closure}()
<a href='/8'>#8</a> /home/user/public_html/analytics.example.com/core/CronArchive.php(319): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/9'>#9</a> /home/user/public_html/analytics.example.com/plugins/CoreConsole/Commands/CoreArchiver.php(27): Piwik\CronArchive->main()
<a href='/10'>#10</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/11'>#11</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/12'>#12</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/13'>#13</a> [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/14'>#14</a> /home/user/public_html/analytics.example.com/core/Console.php(90): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/15'>#15</a> /home/user/public_html/analytics.example.com/core/Access.php(626): Piwik\Console->Piwik\{closure}()
<a href='/16'>#16</a> /home/user/public_html/analytics.example.com/core/Console.php(91): Piwik\Access::doAsSuperUser(Object(Closure))
<a href='/17'>#17</a> /home/user/public_html/analytics.example.com/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
<a href='/18'>#18</a> /home/user/public_html/analytics.example.com/console(27): Symfony\Component\Console\Application->run()
<a href='/19'>#19</a> {main}
@OliverTempel commented on January 14th 2019

Same Error Here.

@OliverTempel commented on January 14th 2019

MaxMind has changed downloads.
New Files has MaxMind Database Format, so no .dat file is included in download.

@sgiehl commented on January 14th 2019 Member

GeoIP Legacy Lite has been announced to be discontinued about a year ago. Since two weeks there are no downloads available anymore. If you wish to use a free database you need to switch to GeoIP 2. To do this you may need to activate the GeoIp2 plugin in Matomo and configured it in the location provider settings.

will close this issue as we won't/can't fix that anymore

refs #13957

This Issue was closed on January 14th 2019
Powered by GitHub Issue Mirror