@diosmosis opened this Issue on December 18th 2018 Member

Currently when purging dangling references from the log_action table, we lock it in order to make sure we don't accidentally delete a row that ends up being referenced during tracking. This is problematic, since purging log_action can take a long time, and blocks tracking while it is running.

The proposed solution is to remove the need for locking by:

  • adding a new action type, ACTION_DELETED. the tracker cannot use an action whose type is ACTION_DELETED
  • when purging, running an update to set the type of all dangling actions to ACTION_DELETED. then running queries to delete all ACTION_DELETED actions.

Issues w/ concurrency must be reviewed carefully when implementing. Eg, we must ensure situations like the following do not occur:

  • update query starts, one action is found to be dangling
  • tracker gets a request for that action, decides to use it
  • update query sets type to ACTION_DELETED
  • tracker uses the idaction, which gets deleted resulting in a broken log

or:

  • tracker gets a request for an action that is currently dangling, and decides to use it
  • update query sees that it is dangling before a log is inserted and sets type to ACTION_DELETED
  • tracker inserts log using idaction which gets deleted resulting in a broken log
@EreMaijala commented on September 27th 2019

Which issue is this a duplicate of?

@tsteur commented on September 29th 2019 Member

@EreMaijala unfortunately I can't find the issue right now.

@diosmosis do you maybe remember?

@diosmosis commented on September 29th 2019 Member

@tsteur No, it was mentioned in slack but we don't have a paid account so we can't look that far back.

@EreMaijala commented on October 2nd 2019

@diosmosis, @tsteur Well, may I suggest reopening this one until the actual duplicate issue is found?

@tsteur commented on October 2nd 2019 Member

Sure 👍

@mattab commented on July 23rd 2020 Member

Regularly people report issues with this locking mechanism. It would make Matomo that bit more stable if log_action deletion required no table locking. The workaround currently is to run the process less often for example once a year with delete_logs_unused_actions_schedule_lowest_interval = 365 or once a month. But that only postpones the problem or make it happen less often. (Also deleting unused actions less often means that the table has more data, and it takes longer to insert into it, update indexes, makes tracking a little slower)

Powered by GitHub Issue Mirror