We're using Piwik with quite some success on many sites, even some high traffic ones. But after adding another high traffic site, we are running into a problem which I can best describe using a quote from "top":
top - 01:22:56 up 5 days, 8 min, 5 users, load average: 1,05, 1,05, 1,05 Tasks: 527 total, 2 running, 525 sleeping, 0 stopped, 0 zombie %Cpu(s): 16,1 us, 0,7 sy, 0,0 ni, 83,0 id, 0,0 wa, 0,0 hi, 0,1 si, 0,0 st Kb Mem: 8197540 total, 7882444 used, 315096 free, 22284 buffers Kb Swap: 5794812 total, 0 used, 5794812 free, 77860 cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15862 www-data 0 -20 5920m 5,4g 11m R 100,2 69,4 174:58.06 php5-fpm
As you can see, the archive.php task is eating up LOTS of memory using LOTS of CPU and taking LOTS of time to do its job. This is with Piwik 1.7.2-rc4. We already tried most if not all suggestions available on the forums and most about everywhere else - problem persists. If anyone is willing to look into this, help would be highly appreciated. Even SSH access to the system would be no problem.
System is a Debian Squeeze install with PHP 5.3.10 FPM and Apache 2.2.22 (worker) as frontend. Timeouts for FastCGI and FPM have been set to 10800 already to make up for the long run time but we're unable to complete at least one run of the today archive in that timeframe. The problem seems to be with the PHP task for the MySQL server is more or less idle during these runs.
Thanks for the report. archive.php should be very cheap memory wise, memory is used by the API requests that are trying to archive the data. How many visits and pageviews do you have on your sites?
I guess I will mark as duplicate of #766
It would be really interesting if you could install XHProf and give me access, as per my comment in: #766
XHProf is set up, I could give you access to the system using Teamviewer right away if that's good enough for you.
Thanks that was fast! Are the generated reports interesting?
Is it possible to email me matt@piwik the URL to access xhprof reports? I don't think teamviewer should be needed for that but let me know.
@whocares any update on the XHprof or the bug? Is there still aproblem in your piwik or can we close the ticket?
As I wrote in my email to you: since your schedule didn't allow for looking into this issue the client went back to using Adsense and removed Piwik from his setup completely. So the ticket can be closed for there is no more data to look at.