Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove invalid hosts tracker settings cache #19021

Merged
merged 7 commits into from Apr 12, 2022
Merged

Conversation

samjf
Copy link
Contributor

@samjf samjf commented Mar 30, 2022

Description:

Ref: DEV-2583

When a tracking request supplies and invalid host an empty folder structure will be made in the cache directory if the GLOBAL ENABLE_CONFIG_PHP_CACHE is true. This change removes cleans up the directory once we can determine it is an invalid host.

Notes
Ideally we wouldn't even create this structure to begin with, however, the vendor lib we use for the file cache throws an exception if a directory path isn't supplied. It was easier just to clean up in the case we have an invalid host after the fact. For the time being this will solve arbitrary dirs stacking up in the tmp folder.

Review

@samjf samjf added the Needs Review PRs that need a code review label Mar 30, 2022
}
}
// remove invalid host dir
rmdir($targetDir);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess this actually does the same as Filesystem::unlinkRecursive()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sgiehl Thanks, I hadn't seen this one 🤦 Though, it doesn't do exactly what we want as it turns out. I wanted this method to be cautious and not delete anything unless it is an empty dir. It seems unlinkRecursive would remove files inside the dirs too.

Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we maybe avoid the path creation at all if we make the valid host check a static method and move the cache creation within that check?

@tsteur
Copy link
Member

tsteur commented Mar 31, 2022

@sgiehl AFAIK we can't because we first need to look into the config to see if it's a valid host before we can know it is a valid host. Although thinking about it now, when using Matomo in a multi tenancy environment where all the config files are in misc/user/$host there would be a way by checking if the userSettingsFile exists. If a config file for a specific host exists, then we can assume it's a valid host. This may be good enough maybe. It be only an issue when using a regular config/config.ini.php. If I see this right then ENABLE_CONFIG_PHP_CACHE is not documented so this could work

diff --git a/core/Config/IniFileChain.php b/core/Config/IniFileChain.php
index 2801cbeb2..970466197 100644
--- a/core/Config/IniFileChain.php
+++ b/core/Config/IniFileChain.php
@@ -215,7 +215,7 @@ class IniFileChain
             $this->resetSettingsChain($defaultSettingsFiles, $userSettingsFile);
         }
 
-        $hasAbsoluteConfigFile = !empty($userSettingsFile) && strpos($userSettingsFile, DIRECTORY_SEPARATOR) === 0;
+        $hasAbsoluteConfigFile = !empty($userSettingsFile) && strpos($userSettingsFile, DIRECTORY_SEPARATOR) === 0 && is_file($userSettingsFile);
         $useConfigCache = !empty($GLOBALS['ENABLE_CONFIG_PHP_CACHE']) && $hasAbsoluteConfigFile;

@samjf
Copy link
Contributor Author

samjf commented Mar 31, 2022

@tsteur That looks to be a good solution. I'm not sure if we could put it on that line of code because reload seems to be the main way to save a missing cache and $useConfigCache needs to be true for that to happen in a normal situation.

We could add the same check just below:

diff --git a/core/Config/IniFileChain.php b/core/Config/IniFileChain.php
index bc37e84ef3..49efb2f71d 100644
--- a/core/Config/IniFileChain.php
+++ b/core/Config/IniFileChain.php
@@ -218,7 +218,7 @@ class IniFileChain
         $hasAbsoluteConfigFile = !empty($userSettingsFile) && strpos($userSettingsFile, DIRECTORY_SEPARATOR) === 0;
         $useConfigCache = !empty($GLOBALS['ENABLE_CONFIG_PHP_CACHE']) && $hasAbsoluteConfigFile;
 
-        if ($useConfigCache) {
+        if ($useConfigCache && is_file($userSettingsFile)) {
             $cache = new Cache();
             $values = $cache->doFetch(self::CONFIG_CACHE_KEY);

@tsteur
Copy link
Member

tsteur commented Mar 31, 2022

I'm not too much into it but would think it pretty much does the same in the end 👍

@samjf samjf removed the Needs Review PRs that need a code review label Apr 3, 2022
@samjf samjf added the Needs Review PRs that need a code review label Apr 4, 2022
@samjf
Copy link
Contributor Author

samjf commented Apr 4, 2022

I've stopped the cache from being created in two places in the IniChainSettings code now. The first one was ensuring that the cache being retrieved actually exists as a file. The second one is ensuring that the merged settings has a valid host setting present/set in the config.
This was close to your suggestion @sgiehl , but it doesn't require actually loading the config (as the code does), just ensuring the config structure is as expected which turns out to be enough.

@samjf samjf removed the Needs Review PRs that need a code review label Apr 5, 2022
@samjf
Copy link
Contributor Author

samjf commented Apr 5, 2022

Appears I have a few tests that need fixing:
Piwik\Tests\Unit\Config\IniFileChainCacheTest::test_reload_canReadFromCache with data set #0
and
Piwik\Tests\Unit\Config\IniFileChainCacheTest::test_reload_canReadFromCache with data set #1

@samjf
Copy link
Contributor Author

samjf commented Apr 6, 2022

I needed to update the test so that it didn't delete the original settings file when testing the cache. That step shouldn't be necessary to test that the cache is indeed being read; a new IniFileChain instance should suffice.

However, this will have an affect, if an original INI settings file is deleted then it will not attempt to read the cache for it (due to $userSettingsFile existence check in reload()). This will effectively invalidate it I believe. I didn't think this would be a problem, but let me know if it is troublesome.

The only other way to proceed would be the original cleanup method that originally was built.

@samjf samjf added the Needs Review PRs that need a code review label Apr 6, 2022
@samjf samjf requested a review from sgiehl April 6, 2022 03:25
Copy link
Member

@sgiehl sgiehl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had a look through the changes. For me this looks good now. Can't think of a reason why this may cause issues. Will merge it once the tests ran through

@sgiehl sgiehl merged commit 95b9d8e into 4.x-dev Apr 12, 2022
@sgiehl sgiehl deleted the D2682-rm-invalid-cache-dirs branch April 12, 2022 12:58
@sgiehl sgiehl removed the Needs Review PRs that need a code review label Apr 12, 2022
@sgiehl sgiehl added this to the 4.10.0 milestone Apr 12, 2022
@sgiehl sgiehl added the Stability For issues that make Matomo more stable and reliable to run for sys admins. label Apr 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Stability For issues that make Matomo more stable and reliable to run for sys admins.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants