Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve purge log data logic to improve speed, and avoid locks in some cases #17632

Merged
merged 2 commits into from May 30, 2021

Conversation

tsteur
Copy link
Member

@tsteur tsteur commented May 27, 2021

Description:

see https://stackoverflow.com/questions/23193761/delete-operation-locks-whole-table-in-innodb/23194154

Deletes may be faster or less locking when they are in descending order. Also added a transaction uncommitted in the hope for this to be less locking and making it faster. This should be fine because we usually delete only older visits anyway and they wouldn't be included in current reports or something.

Problem was that we noticed some locks around the time when the delete visits statement ran even though InnoDB was used.

Review

  • Functional review done
  • Potential edge cases thought about (behavior of the code with strange input, with strange internal state or possible interactions with other Matomo subsystems)
  • Usability review done (is anything maybe unclear or think about anything that would cause people to reach out to support)
  • Security review done see checklist
  • Code review done
  • Tests were added if useful/possible
  • Reviewed for breaking changes
  • Developer changelog updated if needed
  • Documentation added if needed
  • Existing documentation updated if needed

@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label May 27, 2021
@tsteur tsteur added this to the 4.4.0 milestone May 27, 2021
@tsteur tsteur added Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. Needs Review PRs that need a code review and removed Pull Request WIP Indicates the current pull request is still work in progress and not ready yet for a review. labels May 27, 2021
@diosmosis diosmosis merged commit 0e34030 into 4.x-dev May 30, 2021
@diosmosis diosmosis deleted the uncommittedpurge branch May 30, 2021 18:51
@mattab mattab changed the title Improve purge log data logic Improve purge log data logic to improve speed, and avoid locks in some cases Jul 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo. Needs Review PRs that need a code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants