New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When the core:archive script dies with Mysql server has gone away
, can we reconnect to DB automatically instead of erroring?
#7669
Comments
It seems like, for every site two mysql connections are established. One solution could be eliminating the idle connection. |
So as I understand it, the error is "harmless" in the sense that the user can just repeatedly re-run the console archive command until the thing succeeds? There will be no incomplete/corrupt data? As it stands, the official technique involving "console core:archive" does not work at all on shared hosts like Hostgator's; on mine, I am unable to process piwik tracking logs on the commandline through SSH even for some fairly low/medium traffic websites (the piwik instance I'm currently looking at only has 38 thousand rows of tracking log tables, weighting about 6.4 MB). From what I understood of my discussions with HG's support staff, MySQL processes get killed when they use too much CPU time for too long. |
FWIW, I received this additional piece of info/suggestion from Hostgator today:
I don't know if that is a valid suggestion, and if it is, if it should be filed as a separate issue? |
@nekohayo The core:archive already will trigger a separate process for each of the archiving requests. so this should already be implemented |
as @SR-mkuhn pointed out, the common workaround to solve this issue (in most cases) is to increase |
You can't actually do that as a user of a shared host, no? |
@mattab, but wouldn't it be better to work with one mysql connection instead of two AND requirement of increasing wait_timeout? |
Maybe I can give you some more data to help you? I have a problem after upgrade from 2.12.0 to 2.14.3, it goeas away after archiving one website and switching to another. wait_timeout=28800 (it's not the case) It's new archive (droped archive tables) with 4,4k websites and 40+GB piwik_log_visit. |
Hi, As described by @SR-mkuhn , core:archive seems to use two mysql connections. My recommendation is: Greetings |
Looks good. |
Hi @monkz, Thanks in advance, Twan |
Hi @twanmeijerink , My solution is just a recommendation/suggestion for @mattab and @tsteur . |
I solved a similar problem recently.
|
This is not just a matter of wait_timeout. I'm trying to archive 125 days of traffic on a lot of low-traffic sites, because I'm migrating to a new Piwik instance (long story). This causes MySQL to kill the connection. If the number of days is set to 125, then I usually get a failure within 30 seconds of the start of the archiving process. I haven't figured out if the error is because of the size of the received packet, or perhaps even the number of rows to insert is too large. From https://dev.mysql.com/doc/refman/5.7/en/gone-away.html :
My wait_timeout appears to be set to 8 hours. I see the failures when the archiver is working on sites with more traffic, but the sites in question only have ~ 3000 visits in the last couple months. That doesn't seem like a lot of data to process at a time. Regardless of the cause, it would be great if the archiver recovered from this error. |
Thanks everyone for your feedback. We will very likely not implement this in the next few months so I'm closing this issue as |
During
core:archive
cron execution, it can happen that Mysql server fails and then thecore:archive
script itself will die because the connection to database was cut. What this happens the script has to be restarted manually or it will wait until the next cron run. This causes complications to Piwik administrators and we want to make the script smarter, so it tries to re-establish connection to Mysql in case the db server had only a temporary problem.Issue context
Imagine a long running
core:archive
script run. It triggers a lot of requests to the API to pre-process websites. Often a Mysql server may fail and return an error such asMysql server has gone away
. When this happens, the API request will fail and this failure be displayed in thecore:archive
script output, which will proceed with the next request. This works fine and the script continues with the remaining websites..The problem occurs when the console command connection to Mysql has itself been broken. then
core:archive
will die any remaining websites and segments will not be pre-processed.Reproduce
To reproduce: during a
core:archive
run, restart the Mysql server.the
core:archive
script will error with an error:Here is an error that happened in production Piwik:
Proposed solution
The goal is to prevent such error and proceeding with archiving of remaining websites.
The text was updated successfully, but these errors were encountered: