Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Occasional timeout for core:archive task running via fast-cgi #7196

Closed
fhemberger opened this issue Feb 13, 2015 · 5 comments
Closed

Occasional timeout for core:archive task running via fast-cgi #7196

fhemberger opened this issue Feb 13, 2015 · 5 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Milestone
Long term

Comments

@fhemberger
Copy link
Contributor

I'm running piwik on a shared hosting, where PHP is executed via fast-cgi. Occasionally, I'm getting the following error message when executing the core:archive script as cronjob:

ERROR CoreConsole[2015-02-12 22:00:33] Got invalid response from API request: http://example.com/index.php?module=API&method=API.getDefaultMetricTranslations&format=original&serialize=1&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. Please check your Web server Error Log file for more details.
ERROR CoreConsole[2015-02-12 22:00:33] The Piwik URL http://example.com/index.php does not seem to be pointing to a Piwik server. Response was ''.
ERROR CoreConsole[2015-02-12 22:00:33] The Piwik URL http://example.com/index.php does not seem to be pointing to a Piwik server. Response was ''.

This doesn't seem to be related to the memory limit in any way (256MB) for two low traffic sites, I think its because the server sometimes needs a bit more time to spin up the php process via fast-cgi. It would be great if there was an additional option to set a timeout for those requests (I couldn't find it in the core:archive description as force-timeout-for-periods is not related to this issue).
@mattab
Copy link
Member

mattab commented Feb 19, 2015

Hi @fhemberger

in Admin> System check, what text do you get next to "CLI process" ?

(in the code we set CURLOPT_CONNECTTIMEOUT and CURLOPT_TIMEOUT to 0 which should mean "no timeout" but maybe it does not work)

@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Feb 19, 2015
@mattab mattab added this to the Short term milestone Feb 19, 2015
@fhemberger
Copy link
Contributor Author

It says "Not supported (optional)". But the cron itself usually works fine, this error comes up only occasionally.

@mattab mattab modified the milestones: Short term, Mid term Apr 7, 2015
@mattab
Copy link
Member

mattab commented Sep 11, 2015

Hi @fhemberger do you still get this problem? if so, could you paste your core:archive log line Got invalid response from API but also the previous couple lines? i'd like to see how long it takes before timing out (by looking at timestamp in start of the line) and see if maybe it hits some default HTTP timeouts (eg. 30s)...

@fhemberger
Copy link
Contributor Author

The last one I got was on August, 4th, happened only four times since December 2014.
So it really seems to be an edge case … thanks a lot you're still looking into this!

INFO CoreConsole[2015-08-04 15:00:02] ---------------------------
INFO CoreConsole[2015-08-04 15:00:02] INIT
INFO CoreConsole[2015-08-04 15:00:02] Running Piwik 2.14.2 as Super User
INFO CoreConsole[2015-08-04 15:00:02] ---------------------------
INFO CoreConsole[2015-08-04 15:00:02] NOTES
INFO CoreConsole[2015-08-04 15:00:02] - If you execute this script at least once per hour (or more often) in a crontab, you may disable 'Browser trigger archiving' in Piwik UI > Settings > General Settings.
INFO CoreConsole[2015-08-04 15:00:02]   See the doc at: http://piwik.org/docs/setup-auto-archiving/
INFO CoreConsole[2015-08-04 15:00:02] - Reports for today will be processed at most every 10 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO CoreConsole[2015-08-04 15:00:02] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO CoreConsole[2015-08-04 15:00:02] - Archiving was last executed without error 59 min 23s ago
INFO CoreConsole[2015-08-04 15:00:08] - Will process 2 websites with new visits since 59 min 23s , IDs: 1, 2
INFO CoreConsole[2015-08-04 15:00:11] ---------------------------
INFO CoreConsole[2015-08-04 15:00:11] START
INFO CoreConsole[2015-08-04 15:00:11] Starting Piwik reports archiving...
INFO CoreConsole[2015-08-04 15:00:11] Will pre-process for website id = 1, period = day, date = last2
INFO CoreConsole[2015-08-04 15:00:11] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:20] Will pre-process for website id = 2, period = day, date = last2
INFO CoreConsole[2015-08-04 15:00:20] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:36] Archived website id = 2, period = day, 0 segments, 0 visits in last last2 days, 0 visits today, Time elapsed: 0.002s
INFO CoreConsole[2015-08-04 15:00:36] Will pre-process for website id = 2, period = week, date = last2
INFO CoreConsole[2015-08-04 15:00:36] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:39] Archived website id = 2, period = week, 0 segments, 399 visits in last last2 weeks, 76 visits this week, Time elapsed: 2.614s
INFO CoreConsole[2015-08-04 15:00:39] Will pre-process for website id = 2, period = month, date = last2
INFO CoreConsole[2015-08-04 15:00:39] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:41] Archived website id = 2, period = month, 0 segments, 1851 visits in last last2 months, 123 visits this month, Time elapsed: 2.058s
INFO CoreConsole[2015-08-04 15:00:41] Will pre-process for website id = 2, period = year, date = last2
INFO CoreConsole[2015-08-04 15:00:41] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:47] Archived website id = 2, period = year, 0 segments, 18798 visits in last last2 years, 7329 visits this year, Time elapsed: 5.821s
INFO CoreConsole[2015-08-04 15:00:47] Will pre-process for website id = 2, period = range, date = last30
INFO CoreConsole[2015-08-04 15:00:47] - pre-processing all visits
INFO CoreConsole[2015-08-04 15:00:49] Archived website id = 2, period = range, 0 segments, 1728 visits in last last30 ranges, 1728 visits this range, Time elapsed: 2.425s
INFO CoreConsole[2015-08-04 15:00:49] Archived website id = 2, 5 API requests, Time elapsed: 29.776s [2/2 done]
INFO CoreConsole[2015-08-04 15:00:50] Done archiving!
INFO CoreConsole[2015-08-04 15:00:50] ---------------------------
INFO CoreConsole[2015-08-04 15:00:50] SUMMARY
INFO CoreConsole[2015-08-04 15:00:50] Total visits for today across archived websites: 40
INFO CoreConsole[2015-08-04 15:00:50] Archived today's reports for 1 websites
INFO CoreConsole[2015-08-04 15:00:50] Archived week/month/year for 1 websites
INFO CoreConsole[2015-08-04 15:00:50] Skipped 2 websites: no new visit since the last script execution
INFO CoreConsole[2015-08-04 15:00:50] Skipped 0 websites day archiving: existing daily reports are less than 10 seconds old
INFO CoreConsole[2015-08-04 15:00:50] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
INFO CoreConsole[2015-08-04 15:00:50] Total API requests: 5
INFO CoreConsole[2015-08-04 15:00:50] done: 1/2 50%, 40 vtoday, 1 wtoday, 1 wperiods, 5 req, 38418 ms, 2 errors.
INFO CoreConsole[2015-08-04 15:00:50] Time elapsed: 38.418s
INFO CoreConsole[2015-08-04 15:00:50] ---------------------------
INFO CoreConsole[2015-08-04 15:00:50] SCHEDULED TASKS
INFO CoreAdminHome[2015-08-04 15:00:50] Starting Scheduled tasks... 
INFO CoreAdminHome[2015-08-04 15:00:50] done
INFO CoreConsole[2015-08-04 15:00:50] ---------------------------
INFO CoreConsole[2015-08-04 15:00:50] ---------------------------
INFO CoreConsole[2015-08-04 15:00:50] SUMMARY OF ERRORS
INFO CoreConsole[2015-08-04 15:00:50] Error: Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. Please check your Web server Error Log file for more details.
INFO CoreConsole[2015-08-04 15:00:50] Error: Empty or invalid response '' for website id 1, Time elapsed: 8.280s, skipping
INFO CoreConsole[2015-08-04 16:00:15] ---------------------------
...

@fhemberger
Copy link
Contributor Author

Closing this issue, since it didn't appear ever since.
Using latest versions of PHP7 and Piwik.

@mattab mattab added the worksforme The issue cannot be reproduced and things work as intended. label Aug 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Projects
None yet
Milestone
Long term
Development

No branches or pull requests
2 participants
@fhemberger @mattab