@tsteur opened this Pull Request on December 15th 2022 Member

Description:

On production, around 1 in say 10 million requests we have an issue that the MySQL DB connection fails with "MySQL Server has gone away" error.

We've spend many days trying to find root cause and what's happening going down to network layers etc but there doesn't really seem any solution. All MySQL settings etc are in a good place.

Generally, the solution is to retry the connection when it fails. Below patch we've been running on production for some time and it solves the problem that when on establishing the connection we get such an error, that we retry establishing the connection once.

We still have such MySQL gone away errors in other places but this is a start to fix the ones visible in the UI and we'll later may provide few other patches.

The error we're getting was for example:

SQLSTATE[HY000]: General error: 2006 MySQL server has gone away : #0 core/Db/Adapter/Pdo/Mysql.php(109): PDO->exec('SET sql_mode = ...')

See CLOUD-1094 for more details.

Review

@sgiehl commented on December 19th 2022 Member

@tsteur The code looks fine for me. But I was wondering if that might be a problem that could occur with mysqli as well? If so we could maybe add the same try/catch in the mysqli adapters.

@tsteur commented on December 20th 2022 Member

We're not sure if it's an issue there and wouldn't want to make things more complex unless really needed. It be bit more work to test and maintain for edge cases. Like these issues aren't really a problem unless you have a lot of traffic

@tsteur commented on December 20th 2022 Member

Let me know if it's ok to merge @sgiehl

This Pull Request was closed on December 20th 2022
Powered by GitHub Issue Mirror