@mattab opened this Issue on July 12th 2011 Member

We can safely multithread archive.sh script, by running different website ID processing on multiple cores. We can not run archiving for a same website on multiple core, but it will improve performance to run different website ID processing at the same time.

It will improve performance especially in the case of a Piwik install with hundreds/thousands of small/medium websites.

@cbay commented on July 12th 2011 Contributor
@cbay commented on July 12th 2011 Contributor

Here is a first version against trunk. Not compatible with #2440, and I'm not sure it's possible to have both features at the same time, at least with the approach I took (using xargs).

@mattab commented on July 15th 2011 Member

See also #2327

@julienmoumne commented on September 13th 2011 Member

I've read in several tickets related to this one that multi-threading should not be activated on mono core.

Please correct me if I'm wrong but during archiving there is a lot of communication done with the database. As far as I understand scheduling, the scheduler is supposed to switch threads when they are waiting on resources (ie. network and database). During that time, threads needing CPU time (ie. for calculations and aggregations) are given CPU time.

Does it logically follow that activating multi-threading on a single core would increase performance in case the web server and the database are located on different hardware ?

@mattab commented on September 13th 2011 Member

I think considering the huge work done on the new archive.php, we should completely deprecate archive.sh and .ps1 and change all doc to use archive.php and only maintain this one.

This Issue was closed on September 13th 2011
Powered by GitHub Issue Mirror