Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Archiving status: Reexpire lock only when needed #15747

Closed
tsteur opened this issue Mar 30, 2020 · 4 comments · Fixed by #15749
Closed

Archiving status: Reexpire lock only when needed #15747

tsteur opened this issue Mar 30, 2020 · 4 comments · Fixed by #15749
Assignees
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Milestone

Comments

@tsteur
Copy link
Member

tsteur commented Mar 30, 2020

@diosmosis I don't 100% remember anymore why the ArchivingStatus lock was needed?

However, I see we often get heaps of lock timeouts. And these locks often take a considerable amount of load on our DB servers. Eg
image

Often even more than any other query. Often causing a lot of lock wait timeout, and potentially even deadlocks.

Should this lock really still be needed (looked at code again and wasn't sure where it was actually used).
Maybe in ArchivingDbAdapter it could be a start to not reexpire the lock on every DB query but maybe every 10 minutes or 30 minutes considering the lock is valid for up to 7200s by default?

@tsteur tsteur added the c: Performance For when we could improve the performance / speed of Matomo. label Mar 30, 2020
@tsteur tsteur added this to the 3.13.5 milestone Mar 30, 2020
@diosmosis
Copy link
Member

It's needed for #15117 I use it more heavily there. Will look at this soon and coment...

@diosmosis
Copy link
Member

I think we could maybe wait up to, say, have timeout value? Ie, if the timeout is 2 hours, then after an hour, we re-expire?

@diosmosis
Copy link
Member

Actually, wasn't there a timeout for archiving queries? We could use that, if we're N minutes away from expiring or, less, re-expire the lock.

@tsteur
Copy link
Member Author

tsteur commented Mar 30, 2020

Actually, wasn't there a timeout for archiving queries

Not sure what you mean? We have live_query_max_execution_time but this affects only live queries.

Yeah should the TTL be 2 hours, re-expiring say every 30 min be fine (25% of TTL)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c: Performance For when we could improve the performance / speed of Matomo.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants