Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GeoIP: GeoLite2-City.mmdb.broken #13926

Closed
theodorius123 opened this issue Jan 2, 2019 · 16 comments
Closed

GeoIP: GeoLite2-City.mmdb.broken #13926

theodorius123 opened this issue Jan 2, 2019 · 16 comments

Comments

@theodorius123
Copy link

theodorius123 commented Jan 2, 2019 •
edited

Using 3.7.0 I have the problem that my GeoIP-database ("GeoLite2-City.mmdb") is renamed to "GeoLite2-City.mmdb.broken" every few days. And of course GeoIP is not working anymore.

Any ideas what to do? Any information you need to know?

Thanks,
Theo
@sgiehl
Copy link
Member

sgiehl commented Jan 3, 2019

Did you set up and automatic update? Maybe the update url you configured is incorrect and it downloads a broken file resulting in a rename as the file doesn't work

@theodorius123
Copy link
Author

theodorius123 commented Jan 3, 2019 •
edited

Yes, and I used the given URL in the matomo-explanation.
I just disabled this option...lets see if the problem will come back.

thanks!!!

@fdellwing
Copy link
Contributor

What update url do you have setup? https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz should work fine.

@theodorius123
Copy link
Author

Thats is exactly the URL I used. I just copy&pasted the given URL from the description in the settings-menu.

@lourdas
Copy link
Contributor

lourdas commented Jan 9, 2019

This is the error I get during the cron job:

ERROR [2019-01-09 11:50:09] 1264  /*****/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(167): GeoIPAutoUpdater: failed to unzip '/*****/analytics/misc/GeoIPCity.dat.tar.gz' after downloading 'https://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz': Cannot find GeoIPCity.dat file in tar archive '/*****/analytics/misc/GeoIPCity.dat.tar.gz'!
#0 /*****/analytics/plugins/UserCountry/GeoIPAutoUpdater.php(102): Piwik\Plugins\UserCountry\GeoIPAutoUpdater->downloadFile('loc', 'https://geolite...')
#1 [internal function]: Piwik\Plugins\UserCountry\GeoIPAutoUpdater->update(NULL)
#2 /*****/analytics/core/Scheduler/Scheduler.php(246): call_user_func(Array, NULL)
#3 /*****/analytics/core/Scheduler/Scheduler.php(136): Piwik\Scheduler\Scheduler->executeTask(Object(Piwik\Plugins\UserCountry\GeoIPAutoUpdater))
#4 /*****/analytics/plugins/CoreAdminHome/API.php(58): Piwik\Scheduler\Scheduler->run()
#5 /*****/analytics/core/CronArchive.php(612): Piwik\Plugins\CoreAdminHome\API->runScheduledTasks()
#6 /*****/analytics/core/CronArchive.php(317): Piwik\CronArchive->runScheduledTasks()
#7 /*****/analytics/core/Access.php(626): Piwik\CronArchive->Piwik\{closure}()
#8 /*****/analytics/core/CronArchive.php(319): Piwik\Access::doAsSuperUser(Object(Closure))
#9 /*****/analytics/plugins/CoreConsole/Commands/CoreArchiver.php(27): Piwik\CronArchive->main()
#10 /*****/analytics/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#11 /*****/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /*****/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand(Object(Piwik\Plugins\CoreConsole\Commands\CoreArchiver), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 [internal function]: Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /*****/analytics/core/Console.php(90): call_user_func(Array, Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /*****/analytics/core/Access.php(626): Piwik\Console->Piwik\{closure}()
#16 /*****/analytics/core/Console.php(91): Piwik\Access::doAsSuperUser(Object(Closure))
#17 /*****/analytics/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /*****/analytics/console(27): Symfony\Component\Console\Application->run()
#19 {main}

@sgiehl
Copy link
Member

sgiehl commented Jan 9, 2019

this cron shouldn't run when using GeoIP2, but GeoIP2AutoUpdater instead. Are you uing GeoIP Legacy or GeoIP2 as location provider?

@fdellwing
Copy link
Contributor

fdellwing commented Jan 9, 2019 •
edited

Ok, I think we are getting there..

Cannot find GeoIPCity.dat file in tar archive is the relevant error message. The new plugin does not use .dat files anymore, but .mmdb files and in the downloaded archive is no .dat file.

Are you completely sure, that you are using the correct plugin?

Please provide the output of grep 'Plugins\[\]' /*****/analytics/config/config.ini.php and ls -la /*****/analytics/plugins.

@lourdas
Copy link
Contributor

lourdas commented Jan 9, 2019

Ok, I think I have progress. I disabled the old plugin and enabled the new GeoIP2 module. I also modified a script I use to automatically download the new mmdb format and copy it in the appropriate directory. However, the cron job continues to call GeoIPAutoUpdater.php. How can I fix this one?

@lourdas
Copy link
Contributor

lourdas commented Jan 9, 2019

The ls -la plugins:

ls -la .
total 300
drwxr-xr-x 74 www-data www-data 4096 Nov 25 09:32 .
drwxr-xr-x 12 www-data www-data 4096 Nov 25 09:34 ..
drwxr-xr-x 11 www-data www-data 4096 Nov 19 05:19 Actions
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:19 Annotations
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 API
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:19 BulkTracking
drwxr-xr-x 11 www-data www-data 4096 Nov 19 05:19 Contents
drwxr-xr-x 11 www-data www-data 4096 Nov 19 05:19 CoreAdminHome
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:19 CoreConsole
drwxr-xr-x 15 www-data www-data 4096 Nov 19 05:19 CoreHome
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 CorePluginsAdmin
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 CoreUpdater
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 CoreVisualizations
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 CustomPiwikJs
drwxr-xr-x 13 www-data www-data 4096 Nov 19 05:19 CustomVariables
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 Dashboard
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:19 DBStats
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 DevicePlugins
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 DevicesDetection
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 Diagnostics
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 Ecommerce
drwxr-xr-x 11 www-data www-data 4096 Nov 19 05:16 Events
drwxr-xr-x  2 www-data www-data 4096 Nov 19 05:16 ExampleAPI
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:16 ExampleCommand
drwxr-xr-x  9 www-data www-data 4096 Nov 19 05:19 ExamplePlugin
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:16 ExampleReport
drwxr-xr-x  2 www-data www-data 4096 Nov 19 05:16 ExampleSettingsPlugin
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:16 ExampleTheme
drwxr-xr-x  4 www-data www-data 4096 Nov 19 05:16 ExampleTracker
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:16 ExampleUI
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 ExampleVisualization
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:16 Feedback
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 GeoIp2
drwxr-xr-x 14 www-data www-data 4096 Nov 19 05:19 Goals
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:16 Heartbeat
-rw-r--r--  1 www-data www-data 1216 Jan  9 15:34 .htaccess
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:19 ImageGraph
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 Insights
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 Installation
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 Intl
drwxr-xr-x  4 www-data www-data 4096 Nov 19 05:19 IntranetMeasurable
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 LanguagesManager
drwxr-xr-x 12 www-data www-data 4096 Nov 19 05:19 Live
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 Login
drwxr-xr-x 12 www-data www-data 4096 Nov 19 05:19 Marketplace
drwxr-xr-x  3 www-data www-data 4096 Nov 19 05:16 MobileAppMeasurable
drwxr-xr-x  9 www-data www-data 4096 Nov 19 05:19 MobileMessaging
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:19 Monolog
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 Morpheus
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 MultiSites
drwxr-xr-x  9 www-data www-data 4096 Nov 19 05:19 Overlay
drwxr-xr-x 11 www-data www-data 4096 Nov 19 05:19 PrivacyManager
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 ProfessionalServices
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:16 Provider
drwxr-xr-x  2 www-data www-data 4096 Nov 19 05:19 Proxy
drwxr-xr-x  9 www-data www-data 4096 Nov 19 05:19 Referrers
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 Resolution
drwxr-xr-x  4 www-data www-data 4096 Nov 19 05:16 RssWidget
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 ScheduledReports
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 SegmentEditor
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 SEO
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 SitesManager
drwxr-xr-x 25 www-data www-data 4096 Nov 19 05:19 TagManager
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 Transitions
drwxr-xr-x 15 www-data www-data 4096 Nov 19 05:19 UserCountry
drwxr-xr-x 10 www-data www-data 4096 Nov 19 05:19 UserCountryMap
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 UserId
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:19 UserLanguage
drwxr-xr-x  8 www-data www-data 4096 Nov 19 05:19 UsersManager
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 VisitFrequency
drwxr-xr-x  5 www-data www-data 4096 Nov 19 05:16 VisitorInterest
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 VisitsSummary
drwxr-xr-x  7 www-data www-data 4096 Nov 19 05:19 VisitTime
drwxr-xr-x  4 www-data www-data 4096 Nov 19 05:16 WebsiteMeasurable
drwxr-xr-x  6 www-data www-data 4096 Nov 19 05:19 Widgetize

@lourdas
Copy link
Contributor

lourdas commented Jan 9, 2019

And the plugins in config.ini.php:

[Plugins]
Plugins[] = "CorePluginsAdmin"
Plugins[] = "CoreAdminHome"
Plugins[] = "CoreHome"
Plugins[] = "WebsiteMeasurable"
Plugins[] = "IntranetMeasurable"
Plugins[] = "Diagnostics"
Plugins[] = "CoreVisualizations"
Plugins[] = "Proxy"
Plugins[] = "API"
Plugins[] = "Transitions"
Plugins[] = "LanguagesManager"
Plugins[] = "Actions"
Plugins[] = "Dashboard"
Plugins[] = "MultiSites"
Plugins[] = "Referrers"
Plugins[] = "UserLanguage"
Plugins[] = "DevicesDetection"
Plugins[] = "Goals"
Plugins[] = "Ecommerce"
Plugins[] = "SEO"
Plugins[] = "Events"
Plugins[] = "UserCountry"
Plugins[] = "GeoIp2"
Plugins[] = "VisitsSummary"
Plugins[] = "VisitFrequency"
Plugins[] = "VisitTime"
Plugins[] = "VisitorInterest"
Plugins[] = "RssWidget"
Plugins[] = "Monolog"
Plugins[] = "Login"
Plugins[] = "UsersManager"
Plugins[] = "SitesManager"
Plugins[] = "Installation"
Plugins[] = "CoreUpdater"
Plugins[] = "CoreConsole"
Plugins[] = "ScheduledReports"
Plugins[] = "UserCountryMap"
Plugins[] = "Live"
Plugins[] = "CustomVariables"
Plugins[] = "PrivacyManager"
Plugins[] = "ImageGraph"
Plugins[] = "Annotations"
Plugins[] = "Overlay"
Plugins[] = "SegmentEditor"
Plugins[] = "Morpheus"
Plugins[] = "Contents"
Plugins[] = "BulkTracking"
Plugins[] = "Resolution"
Plugins[] = "DevicePlugins"
Plugins[] = "Heartbeat"
Plugins[] = "Intl"
Plugins[] = "Marketplace"
Plugins[] = "ProfessionalServices"
Plugins[] = "UserId"
Plugins[] = "CustomPiwikJs"
Plugins[] = "Provider"
Plugins[] = "DoNotTrack"

[PluginsInstalled]
PluginsInstalled[] = "Login"
PluginsInstalled[] = "CoreAdminHome"
PluginsInstalled[] = "UsersManager"
PluginsInstalled[] = "SitesManager"
PluginsInstalled[] = "Installation"
PluginsInstalled[] = "CorePluginsAdmin"
PluginsInstalled[] = "CoreHome"
PluginsInstalled[] = "Proxy"
PluginsInstalled[] = "API"
PluginsInstalled[] = "Widgetize"
PluginsInstalled[] = "Transitions"
PluginsInstalled[] = "LanguagesManager"
PluginsInstalled[] = "Actions"
PluginsInstalled[] = "Dashboard"
PluginsInstalled[] = "MultiSites"
PluginsInstalled[] = "Goals"
PluginsInstalled[] = "SEO"
PluginsInstalled[] = "UserCountry"
PluginsInstalled[] = "VisitsSummary"
PluginsInstalled[] = "VisitFrequency"
PluginsInstalled[] = "VisitTime"
PluginsInstalled[] = "VisitorInterest"
PluginsInstalled[] = "ExampleAPI"
PluginsInstalled[] = "ExamplePlugin"
PluginsInstalled[] = "ExampleRssWidget"
PluginsInstalled[] = "Provider"
PluginsInstalled[] = "Feedback"
PluginsInstalled[] = "CoreUpdater"
PluginsInstalled[] = "UserCountryMap"
PluginsInstalled[] = "Live"
PluginsInstalled[] = "CustomVariables"
PluginsInstalled[] = "PrivacyManager"
PluginsInstalled[] = "ImageGraph"
PluginsInstalled[] = "DoNotTrack"
PluginsInstalled[] = "Annotations"
PluginsInstalled[] = "MobileMessaging"
PluginsInstalled[] = "Overlay"
PluginsInstalled[] = "SegmentEditor"
PluginsInstalled[] = "SecurityInfo"
PluginsInstalled[] = "CoreConsole"
PluginsInstalled[] = "CoreVisualizations"
PluginsInstalled[] = "Referrers"
PluginsInstalled[] = "ScheduledReports"
PluginsInstalled[] = "Events"
PluginsInstalled[] = "Morpheus"
PluginsInstalled[] = "DevicesDetection"
PluginsInstalled[] = "Contents"
PluginsInstalled[] = "BulkTracking"
PluginsInstalled[] = "Resolution"
PluginsInstalled[] = "DevicePlugins"
PluginsInstalled[] = "Ecommerce"
PluginsInstalled[] = "UserLanguage"
PluginsInstalled[] = "Monolog"
PluginsInstalled[] = "Diagnostics"
PluginsInstalled[] = "WebsiteMeasurable"
PluginsInstalled[] = "Intl"
PluginsInstalled[] = "Heartbeat"
PluginsInstalled[] = "UserId"
PluginsInstalled[] = "ProfessionalServices"
PluginsInstalled[] = "CustomPiwikJs"
PluginsInstalled[] = "Marketplace"
PluginsInstalled[] = "RssWidget"
PluginsInstalled[] = "IntranetMeasurable"
PluginsInstalled[] = "GeoIp2"

@lourdas
Copy link
Contributor

lourdas commented Jan 9, 2019

Is it possible to disable the automatic update of the city ip database file?

@sgiehl
Copy link
Member

sgiehl commented Jan 14, 2019

@lourdas if you configure the GeoIP2 Autoupdater in Matomo it will disable the old one.
To disable GeoIP Legacy updates manually you can remove the according options in the database
DELETE FROM matomo_option WHERE option_name like 'geoip.%';

To everyone else. Maxmind has discontinued support for GeoIP Lite Legacy databases. If you wish to continue using a free maxmind database, you need to switch to GeoIP2. Therefor you may need to activate the GeoIp2 plugin in Matomo and configure it in location provider settings.

@sgiehl
Copy link
Member

sgiehl commented Oct 28, 2019

Closing this one for now. If anyone is still having problems with GeoIP2 update please open a new issue.

@sgiehl sgiehl closed this as completed Oct 28, 2019
@gamanet
Copy link

gamanet commented Jan 2, 2020

From https://dev.maxmind.com/geoip/geoip2/geolite2/

Due to upcoming data privacy regulations, we are making significant changes to how you access free GeoLite2 databases starting December 30, 2019. Learn more on our blog.

@kghbln
Copy link

kghbln commented Jan 2, 2020 •
edited

See: #15308

@gamanet
Copy link

gamanet commented Jan 2, 2020

From https://dev.maxmind.com/geoip/geoip2/geolite2/

Due to upcoming data privacy regulations, we are making significant changes to how you access free GeoLite2 databases starting December 30, 2019. Learn more on our blog.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@gamanet @lourdas @theodorius123 @kghbln @sgiehl @fdellwing