Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When log deletion is enabled, do not allow import of any older data #14802

Closed
tsteur opened this issue Aug 22, 2019 · 0 comments · Fixed by #14831
Closed

When log deletion is enabled, do not allow import of any older data #14802

tsteur opened this issue Aug 22, 2019 · 0 comments · Fixed by #14831
Assignees
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Milestone

Comments

@tsteur
Copy link
Member

tsteur commented Aug 22, 2019

Say you configured logs to be deleted after 3 months in the privacy settings, then we don't want to allow the tracking of any data that is older than 3 months as we would delete them again anyway.

We could also add new tracking failure type for this maybe but to be seen.

Kind of similar to #14801 we want to fail early when this is the case. Right after https://github.com/matomo-org/matomo/blob/3.12.0-b1/core/Tracker/Visit.php#L131 we want to get the timestamp, and if a request time is older than a potentially configured log deletion time, then we need to discard the request.

This logic is only performed if log deletion is enabled (delete_logs_enable = 1)

And we discard the tracking request using InvalidRequestParameterException() only if the $request->currentTimestamp() is older than (delete_logs_schedule_lowest_interval + delete_logs_older_than)`. AFAIK both privacy config values are configured in days.

To make this performant we need to make sure the data is read from the cache. We can set the config values for these settings here: https://github.com/matomo-org/matomo/blob/3.12.0-b1/plugins/PrivacyManager/PrivacyManager.php#L205 like (just pseudo code)

$caeche['privacymanager']['delete_logs_enable'] = PurgeDataSettings[...]
$caeche['privacymanager']['delete_logs_schedule_lowest_interval'] = PurgeDataSettings[...]
$caeche['privacymanager']['delete_logs_older_than'] = PurgeDataSettings[...]

And then read them in Visit.php like $cache = Tracker\Cache::getCacheGeneral() and then do like (just pseudo code)

if (!empty($cache['privacymanager']['delete_logs_enable']) && !empty($caeche['privacymanager']['delete_logs_older_than'])){   
$timeDropRequest = time() - (delete_logs_schedule_lowest_interval + delete_logs_older_than) * 60*60*24);
if ($request->getCurrentTimestamp() < timeDropRequest) {}   
}

The log deletion setting can be configured in the privacy manager UI or in the config currently but that's not really relevant for this.

@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label Aug 22, 2019
@tsteur tsteur added this to the 3.12.0 milestone Aug 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants