Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When misc/GeoLiteCity.dat file is missing, this can break Piwik #8489

Closed
mattab opened this issue Aug 5, 2015 · 19 comments
Closed

When misc/GeoLiteCity.dat file is missing, this can break Piwik #8489

mattab opened this issue Aug 5, 2015 · 19 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.

Comments

@mattab
Copy link
Member

mattab commented Aug 5, 2015

Right now, demo.piwik.org shows "This connection was reset" error. It's completely broken and not accessible. Looking at errors logs we can see continuously these errors added:

[Wed Aug 05 15:19:24 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Aug 05 15:19:25 2015] [notice] child pid 510957 exit signal Segmentation fault (11)
Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat

Suggested solution:

  • when the /misc/GeoLiteCity.dat file is corrupted, Piwik / PHP process should not crash
  • can we add a test that simulates a corrupt /misc/GeoLiteCity.dat and check that Piwik tracking + UI still work?
@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Aug 5, 2015
@mattab
Copy link
Member Author

mattab commented Aug 5, 2015

It breaks Tracker too! eg. http://demo.piwik.org/piwik.php?redirecturl=http%3A%2F%2Fplugins.piwik.org%2FCustomOptOut&c_i=click...... does not redirect.

Looking on the server, the file does not actually exist.

$ ls -la /home/piwik-demo/www/demo.piwik.org/misc/Geo*
-rw-r--r-- 1 piwik-demo piwik-demo 19237666 2012-12-21 10:57 /home/piwik-demo/www/demo.piwik.org/misc/GeoIPCity.dat.broken
-rw-r--r-- 1 piwik-demo piwik-demo  2918213 2014-11-05 18:35 /home/piwik-demo/www/demo.piwik.org/misc/GeoIPISP.dat
-rw-r--r-- 1 piwik-demo piwik-demo 32646602 2014-11-05 18:35 /home/piwik-demo/www/demo.piwik.org/misc/GeoIPOrg.dat
-rw-r--r-- 1 piwik-demo piwik-demo 19237666 2012-12-21 10:57 /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat.broken

I'm wondering why Piwik tries to load the file when it does not even exist, maybe this is the bug?

@mattab
Copy link
Member Author

mattab commented Aug 5, 2015

FYI: to workaround this issue I had to re-download the geoip database:

$ wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
$ gunzip GeoLiteCity.dat.gz
$ ls GeoLiteCity.dat -la
-rw-rw-r-- 1 piwik-demo piwik-demo 15418580 2015-08-05 05:46 GeoLiteCity.dat

@mattab mattab changed the title When misc/GeoLiteCity.dat file is corrupted, this can break Piwik When misc/GeoLiteCity.dat file is missing, this can break Piwik Aug 5, 2015
@mattab mattab added the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Aug 5, 2015
@mattab mattab added this to the 2.15.0 milestone Aug 5, 2015
@tsteur
Copy link
Member

tsteur commented Aug 6, 2015

I tried to reproduce by removing the file misc/GeoIP.dat and by corrupting this file (removing lots of content everywhere, replacing it with other files etc). I cannot reproduce it neither in Tracker nor in Piwik. Also tried Default Provider, PHP provider and PECL provider. Even restarted apache etc after each change.

Do you still have that corrupt file? Tried http://demo.piwik.org/misc/GeoIPCity.dat.broken but doesn't work. Which provider is used?

@mattab
Copy link
Member Author

mattab commented Aug 10, 2015

unfortunately we cannot reproduce the issue yet.

If you experience this issue, please comment here!

@mattab mattab removed this from the 2.15.0 milestone Aug 10, 2015
@mattab mattab added this to the 2.15.0 milestone Sep 2, 2015
@mattab
Copy link
Member Author

mattab commented Sep 2, 2015

Again got the issue today. Demo.piwik.org is down, error log shows:

Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:50 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:51 2015] [notice] child pid 3040200 exit signal Segmentation fault (11)
Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:52 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:53 2015] [notice] child pid 3040305 exit signal Segmentation fault (11)
Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:54 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:55 2015] [notice] child pid 3040425 exit signal Segmentation fault (11)
Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:56 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:57 2015] [notice] child pid 3040529 exit signal Segmentation fault (11)
Error Opening file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:58 2015] [error] [mod_geoip]: Error while opening data file /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
[Wed Sep 02 08:19:59 2015] [notice] child pid 3040641 exit signal Segmentation fault (11)

the file is that is being opened does not even exist:

piwik-demo@piwik:~/www/demo.piwik.org$ ls -la /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat
ls: cannot access /home/piwik-demo/www/demo.piwik.org/misc/GeoLiteCity.dat: No such file or directory

It's worrying that when this file is not even there, Piwik is broken in this way. let's investigate!

@tsteur
Copy link
Member

tsteur commented Sep 2, 2015

I investigated and couldn't reproduce that time. Maybe someone else can

@sgiehl
Copy link
Member

sgiehl commented Sep 6, 2015

Do we have mod_geoip for apache on demo.piwik.org? Maybe it crashes if the configured GeoIPDBFile is not found?

@mattab
Copy link
Member Author

mattab commented Sep 8, 2015

Do we have mod_geoip for apache on demo.piwik.org?

Yes mod_geoip for apache is activated on demo

Maybe it crashes if the configured GeoIPDBFile is not found?

Maybe indeed, if you can reproduce let us know!

@sgiehl
Copy link
Member

sgiehl commented Sep 8, 2015 via email

@sgiehl
Copy link
Member

sgiehl commented Sep 8, 2015

I've just installed mod_geoip on my development box. Removing or corrupting does produce the " Error while opening data file" but everything is still working - no segfault.
Which version of apache/mod_geoip is installed?

@mattab
Copy link
Member Author

mattab commented Sep 8, 2015

Apache 2.2

$ apt-cache show libapache2-mod-geoip | grep Version
Version: 1.1.8-2

@mattab mattab modified the milestones: 2.15.1, 2.15.0 Sep 22, 2015
@mattab mattab removed the Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical. label Oct 8, 2015
@mattab mattab modified the milestone: 2.15.1 Oct 20, 2015
@mattab mattab added this to the Mid term milestone Nov 26, 2015
@stijnstijn
Copy link

I am having this exact issue. Running a Piwik install without problems, followed the steps in the FAQ for installing Geolocation via PECL, and now the config page for geolocation won't load and that segmentation fault shows up in the log.

I haven't really been able to find out more about this and the workaround of redownloading the database didn't work for me, but I'll be happy to do anything that may help you debug this.

Interestingly when I comment out the geoip.custom_directory directive in php.ini, everything works again and PECL geolocation works as well. The config page says the directive is empty but I suppose Piwik found the way to the database file on its own.

@raj45
Copy link

raj45 commented Jan 29, 2016

The GeoIPCity.dat was renamed by the system to GeoIPCity.dat.broken, and geotracking stopped working, I don't know when. My Piwik-installation worked fine, though.

I restored a fresh copy of GeoIPCity.dat to the /misc folder today, and everything seems to be working again. I am using the GeoIP (Apache) service.

@raj45
Copy link

raj45 commented Feb 5, 2016

I got the below error by mail, and the location no longer works. Also, GeoIPCity.dat in the /misc folder has been renamed to GeoIPCity.dat.broken ...

ERROR [2016-02-03 00:28:55] GeoIPAutoUpdater: Encountered PHP error when performing redundant tests on GeoIP loc database: 2: include(): Failed opening '/data/www/website/tmp/cache/tracker/piwikcache_Translations-en-ebd64ae248b2308a68b47baeadaf0cc549a66c38.php' for inclusion (include_path='/data/www/website/vendor/pear/pear_exception:/data/www/website/vendor/pear/console_getopt:/data/www/website/vendor/pear/pear-core-minimal/src:/data/www/website/vendor/pear/archive_tar:.:/usr/share/php:/usr/share/pear') on line 112 of /data/www/website/vendor/doctrine/cache/lib/Doctrine/Common/Cache/PhpFileCache.php.

@mattab mattab modified the milestones: 2.16.x, Long term Feb 7, 2016
@mattab
Copy link
Member Author

mattab commented Feb 7, 2016

@sgiehl Maybe you have a chance to look into this issue? we also experience it on demo2, it would be awesome to address it in our LTS 👍

@sgiehl
Copy link
Member

sgiehl commented Feb 7, 2016

I've retried to reproduce, but I'm not able to reproduce the segfault. Maybe it only occurs with a special combinations of os/php/pecl versions.

We maybe should try to reduce to possibility that those files are even missing. If I've seen that correctly, the GeoIPUpdater tries to download and update those files. If a downloaded file doesn't work it will be renambed to *.broken. Why don't we keep a backup of the file that was used before, and reuse that one if the update fails?

@mattab mattab modified the milestones: 2.16.x (LTS), Mid term Apr 1, 2016
@sballinone
Copy link

Same issue for me. If I have a GeoIPCity.dat located in the misc directory, it breaks Piwik. If not, Piwik runs. Ubuntu 16.04, Matomo 3.3.0, php7, geoip1.1.1

@sgiehl
Copy link
Member

sgiehl commented Jan 14, 2019

refs #13957

@sgiehl sgiehl added the wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it. label Dec 2, 2019
@sgiehl
Copy link
Member

sgiehl commented Dec 2, 2019

I will close this issue now. GeoIP was deprecated by maxmind, and thus using the Pecl extension doesn't make much sense anymore, as it doesn't support GeoIP2 databases.

@sgiehl sgiehl closed this as completed Dec 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

6 participants