Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to automatically set up geoip2 in installation #15939

Merged
merged 9 commits into from May 14, 2020

Conversation

sgiehl
Copy link
Member

@sgiehl sgiehl commented May 11, 2020

As discussed in #12811 I've added a new checkbox in the last step of the installation.

When checked, Matomo will automatically add the GeoIP2Updater setting, so the dbip database will be donwloaded/updated monthly. In addition it will set the used location provider to geoip2php.

But I'm currently unsure how to handle the automatic download of the database. Doing that within the installation process is a bit difficult to implement and it also might take a while.

With this implementation the updater will be set and thus the automatic task should be scheduled for "tomorrow". That means until then the geolocation would be reported as broken in admin, as geoip2 is configured, but no database is available. (geolocation itself would still work and fallback to the default provider)
I've changed the geolocation admin a bit, so it will automatic start the download process if autoupdate is configured but no database is available.
Not sure if that is a suitable solution, but didn't have a better idea right now.
@mattab @tsteur @diosmosis Does anyone of you have a better idea how to do/trigger the download. We could maybe try to run it async, but that might not work in many cases 馃

fixes #12811

@sgiehl sgiehl added Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. labels May 11, 2020
@sgiehl sgiehl added this to the 4.0.0 milestone May 11, 2020
@diosmosis
Copy link
Member

what's the issue w/ making the controller requests it to the installation UI?

@sgiehl
Copy link
Member Author

sgiehl commented May 11, 2020

what's the issue w/ making the controller requests it to the installation UI?

That won't work easily with the hooks currently used to add the checkbox I guess.

@diosmosis
Copy link
Member

I see, I guess the original code probably uses angular or a lot of JS that needs to be loaded? Or would need to be modified to be pre-loaded during installation?

@tsteur
Copy link
Member

tsteur commented May 11, 2020

Wouldn't do it directly during installation but only once fully installed. Could we set eg during the installation a flag that we need to download the DB... then we try to complete this download eg as part of an hourly task which would simply be executed eg on the first tracking request or so. Additionally, if CliMulti::supportsAsync() returns true, then we could launch a command in the background to sync the DB... AFAIK supportsAsync() works on many installs and most often it would be downloaded fairly soon after installation is completed (could trigger it on last step for example). For all other cases we'd download it through a task.

Not sure it's somewhat clear what I mean?

@sgiehl sgiehl marked this pull request as ready for review May 12, 2020 09:21
@sgiehl
Copy link
Member Author

sgiehl commented May 12, 2020

@tsteur change that. it will now be done using a scheduled task and if async is possible, the scheduled tasks will be triggered in the last step of installation

Copy link
Member

@tsteur tsteur left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks all good. Left one comment re launching the command. Might be good to look into it.


if ($cliMulti->supportsAsync()) {
$phpCli = new CliMulti\CliPhp();
$command = sprintf('%s %s/console core:run-scheduled-tasks > %s/tmp/out.txt 2>&1 &',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgiehl might be good to run eg console core:run-scheduled-tasks --force "Piwik\Plugins\GeoIp2\GeoIP2AutoUpdater.update" directly? Also is the redirection needed to out.txt needed? Eg on nginx by default the file would maybe be directly accessible. Not a big issue but might just not be needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the output shouldn't be needed, that was for debugging purpose 馃檲
will change that to directly execute the task.

@@ -1,6 +1,8 @@
{
"GeoIp2": {
"AssumingNonApache": "Cannot find apache_get_modules function, assuming non-Apache webserver.",
"AutomaticSetup": "Automatically configure geolocation using a dbip database",
"AutomaticSetupDescription": "For a proper geolocation Matomo requires an external database. Using this option, Matomo will automatically be configured to download an use the latest %1$sdbip%2$s city level database.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could maybe say ... city level database which is licensed under the [Creative Commons Attribution 4.0 International License](https://db-ip.com/db/lite.php?refid=mtm)". or just say ... city level database. [View licensing terms](https://db-ip.com/db/lite.php?refid=mtm). Not sure we do this in the GeoIP downloader already. If not, then it might not be too important. Generally be good though if we do it in the admin as well. fyi @Findus23

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image
sorry just seeing this is actually already done just not as clear.

@sgiehl
Copy link
Member Author

sgiehl commented May 13, 2020

@tsteur updated the code

@tsteur
Copy link
Member

tsteur commented May 13, 2020

Looks good to me 馃憤

@sgiehl sgiehl merged commit dc4c494 into 4.x-dev May 14, 2020
@sgiehl sgiehl deleted the geoip2setup branch May 14, 2020 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Activate DB-Ip support by default for all users
3 participants