There used to be a lock to ensure that a given archive is only processed once.

When 10 widgets requested the same site + date + segment at once, then this lock ensured that only the first widget processes the data, while other are waiting for the lock to be released (and then will just read the archive data)

Although there used to be this lock, the lock did not really work and was removed therefore, see a0acaac

Basically there was this outer GET_LOCK (that's the lock we are talking about) which was immediately released by an inner GET_LOCK. So the outer lock never really worked. In addition, even if the lock worked, there was no check whether data was already archived or not. So if there were 10 requests at the same time they would have still created 10 archive ids.

Link to MySQL doc: https://dev.mysql.com/doc/refman/5.0/en/miscellaneous-functions.html#function_get-lock

Goal of this ticket is to make it work again.