Currently when someone sets up a core:archive crontab, the recommendation is to just simply set it without any additional options: https://matomo.org/docs/setup-auto-archiving/
In a lot of cases this works without any issues. However, if for some reason a particular run of the core:archive takes longer than usual (For example lots of tracking data all of a sudden) or for instances that track a lot data already this may lead to many concurrent archivers running at the same time.
This in turn results in even slower archiving performance and becomes a negative feedback loop eventually resulting in the database server hitting 100% CPU usage.
It would be great if we could set a default limit to the number of concurrently running archivers (Maybe some like a default max of 3 archivers).
I don't think this would have any impact on people not already setting the number of concurrent archivers. The one negative would be that if someone wanted to run a manual core:archive command and there are already core:archive commands running, then this would cause that archiver to exit.
In this case maybe an additional option to disable the check for maximum archivers might work for example. (We could also hint this in the output when the archiver exits for this reason).
One example of how this impacted 2 customers recently: One customer ended up with over 60 core:archive crontabs starting over a period of one or two days. This caused the CPU of the Reader database to hit 100% consistently and none of the core:archivers were able to finish because the database server was already way overloaded with requests.
The other customer did not have a Reader database setup and they ended up with around 25 concurrent archivers running which caused CPU to hit 100% consistently. This prevented them from being able to use Matomo (They were not even able to load the UI)
BTW another solution, without possibly breaking it could be to simply adjust the guide. There will be potentially always few people where it would break something, just the same way as it would be fixing something for few potentially. We could maybe then change the default in Matomo 5 or so.
Sounds good to maybe include this sort of change in a bigger version release. Changing the guide will help people who set it up in the future, but I think by setting a default like this could potentially help quite a few people who already have it setup and maybe don't even realise that sometimes their server is slow for this reason.