Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scheduled task to delete log data for sites that have been deleted. #13227

Merged
merged 4 commits into from Aug 6, 2018

Conversation

diosmosis
Copy link
Member

Changes:

  • Add ability to delete log data completely by idsite to DataSubjects.
  • Add PrivacyManager.deleteDataSubjectsForDeletedSites event.
  • Add weekly scheduled task to delete log data for deleted sites.
  • Add message to remove site dialog about permanent deletion:

screen shot 2018-07-30 at 5 02 13 pm

Fixes #12907

@diosmosis diosmosis added not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Needs Review PRs that need a code review labels Jul 31, 2018
@diosmosis diosmosis added this to the 3.6.0 milestone Jul 31, 2018
Copy link
Member

@mattab mattab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback

  • Could we make the SQL query work with a JOIN so it works well with 10,000 IDs?
  • I propose to remove the event PrivacyManager.deleteDataSubjectsForDeletedSites because instead the easier to understand event SitesManager.deleteSite.end can be used

Then please merge @diosmosis

// an ongoing delete, the new valid data won't be deleted.
$maxIdSite = max($idSites);

$where = "$tableToSelect.idsite NOT IN (" . implode(',', $idSites) . ") AND $tableToSelect.idsite <= ?";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would probably be better here to JOIN to the site table so it works fast when there are 10,000 websites? as I think the NOT IN (10000 ids here) will not work well IIRC

@diosmosis
Copy link
Member Author

I propose to remove the event PrivacyManager.deleteDataSubjectsForDeletedSites because instead the easier to understand event SitesManager.deleteSite.end can be used

What if the plugin has to delete from a custom log table where there is lots of data? Doing that in SitesManager.deleteSite.end would be impractical... Will remove the event though, since it can always be added later.

@diosmosis
Copy link
Member Author

@mattab Updated.

@mattab mattab merged commit ccff212 into 3.x-dev Aug 6, 2018
@mattab mattab deleted the 12907-delete-site-data branch August 6, 2018 14:13
InfinityVoid pushed a commit to InfinityVoid/matomo that referenced this pull request Oct 11, 2018
…d. (matomo-org#13227)

* Add scheduled task to delete log data for sites that have been deleted.

* Update Changelog for new event.

* Use JOIN instead of NOT IN & remove event.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Review PRs that need a code review not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants