I enable geoip2php, download the file and save the settings,
then after at least one week it disables automatically and when navigating to the geo configuration page it shows this error message in red:
The configured geolocation provider geoip2php is not available anymore, please configure another one.
I need then to download the file again and start from beginning.
Hi @mstenz when you go to Matomo => Admin => Plugins, can you check if the GeoIP2 plugin is enabled? If not, can you try to enable it and see if works now?
Yes, the plugin is enabled. Today the settings were again reset. Don't know whats going on. I have enabled the log settings, but there is no error about this inside. This was working since years, but now after the update makes issues.
@mstenz just to correctly understand. You basically need to configure Geolocation every time from scratch and need to configure things like the URL from where to download the file and get Matomo to download it etc?
Any chance you can send us the output of your Matomo system report? You find it in "Matomo => Admin => System check" and the output should be anonymised.
Just enabled debugging and found out that:
geo2ip works fine after first activated
Line 520: DEBUG UserCountry[2020-12-27 21:05:50 UTC]  GEO: Found IP 126.96.36.199 location (provider 'geoip2php'): array (
Line 70971: DEBUG UserCountry[2020-12-28 01:59:51 UTC]  GEO: no current location provider sent, falling back to default 'default' one.
@mstenz is maybe the geoip2 database removed somehow? Can't think of another reason why it should automatically disable the provider 🤔
i have just downloaded the file again. it is stored in misc folder and there is no antivirus (except from Windows Defender) installed on the system that is able to delete the file. i don't think there is any other processes on the system that would potentially delete any file. Let me watch the file and i will give you an update on this.
i found that every day sometimes near midnight the file get deleted. Please see this file action. I found out that when activing geoip2php the first time there is a task created for the next day
DEBUG GeoIp2[2021-01-06 01:46:05 UTC] [60e2a] Rescheduling task and setting first run for tomorrow Piwik\Plugins\GeoIp2\GeoIP2AutoUpdater.update
I have now changed the schedule of this task to run in the next minutes, i assume the file again get deleted.
So it first get deleted, but not recreated, and subsequent accesses to the file then is failing. Unfortunately i had an issue with the debug-log today and there was nothing in at that time. maybe there is an error in the updating script on Windows? Not sure.
Any way I can debug this further?
ok. I found out that the
in GeoIP2AutoUpdater.php arround line 320 is the problem. When i uncomment this line everything works as expected and the file get replaced.
My assumption is, that the file is somehow locked becuase it was access by the php.exe process before. then the copy is done as expected, but because it is marked as "DELETE PENDING", as soon the php.exe process is away the file get deleted.
From the php documentation
On Windows, it is now possible to unlink() files with handles in use, while formerly that would fail. However, it is still not possible to re-create the unlinked file, until all handles to it have been closed.
I think the update process for Windows machines need to be re-thought completely to make it more stable and really working.
Interesting @mstenz thanks for finding this.
@sgiehl do you have a windows with a running Matomo installation maybe? Any chance you could reproduce this? I wonder if eg https://github.com/matomo-org/matomo/blob/4.1.1-b2/core/Http.php#L108 is a problem and keeps the file handle open. But doubt it would be the specific issue cause it should be first downloaded through HTTP into a tmp dir.
But doubt it would be the specific issue cause it should be first downloaded through HTTP into a tmp dir.
If I understand completely the problem is only if the file is exactly the same. It get download to the tmp dir without any problem. But because the $oldDbFile is first deleted and then replaced with new file from tmp directory Windows does not delete it directly when the command is executed, only when php.exe is closed.
first the file get downloaded to tmp, then it uses unlink and the target file get now deleted, then it tries to move the file (which fails) it goes to the next command which copy the file instead, but because move failed windows markes the file as "DELETE PENDING" and as soon php.exe finishes Windows deletes the file again (even this is not wat was expected) . This is what I understand from my debuggings.
But you are right. The http function in core is also faulty as it doesnt close it, its just no problem because the file is not needed after php.exe is finished.
@tsteur No, don't have set up PHP on windows atm. But could try doing that in a VM
Sounds good to like time restrict maybe max 2 hours or so in order to see if we can reproduce it.
Just set up a WAMP environment. Running the geoip update with an existing file always results in a deleted file. There are no other processes accessing the file. So it seems not possible to remove the file within the process. Need to check if there is a simple possibility to circumvent this
The ideas i have for preventing this and make it working both with Linux and windows:
👍 adding it into the milestone
If I understand things correctly it should be maybe fixable that all file handles to it are closed? But not sure where it's left open (maybe the Geo IP lib or so)
as the move fails. i assume there is a bug in php and php itself does not close the handle. with would be possible with fopen and so on to rebuild the move command but I do not think thats worth the effort. in my comment before with the screenshot you see that the file is opened 7 times but only closed 3 times. so there are 4 handles that are not closed.
I think you can do a "Fast workarround" as i have did and just coment out the unlink line as this fixes the problem for both windows and linux.