General error: 2006 MySQL server has gone away

“MySQL Server has gone away” is a cryptic error that can be hard to troubleshoot (look at all the various responses on Stack Overflow!) Many problems can cause this error; I would like to document one specific case. In this example, the client is a PHP app using the Phalcon framework:

[Mon, 09 Apr 18 03:34:08 -0400][ERROR]  SQLSTATE[HY000]: General error: 2006 MySQL server has gone away
exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /path/to/ModelBase.php:
Stack trace:
#0 [internal function]: PDOStatement->execute()
...
#17 {main}

This error can be tricky to troubleshoot. I looked in three logs:

    1. MySQL error log on the server

      1. MariaDB:

        sudo journalctl -u mariadb

      2. MySQL/Percona:

        sudo journalctl -u mysql

    2. MySQL slow query log on the server (may not be enabled on your server)

    3. MySQL client log

No errors or slow queries happened on the MySQL server at the time that it “went away;” clearly the server was not actually crashing or dying. The key to troubleshooting this error is to look at the other errors that are logged before and/or after it on the SQL client. The key phrase is “Got a packet bigger than ‘max_allowed_packet’ bytes”, which means that the client is sending a single chunk of data that’s greater than what the SQL server is configured to accept. In our case, such an error should not have happened, because the PHP application “batches” data for processing. However, by following the stacktrace, we discovered that a developer was also writing raw, un-batched data to the DB for debugging purposes. As soon as we disabled that process, the errors stopped.

[Mon, 09 Apr 18 03:34:08 -0400][ERROR]  SQLSTATE[08S01]: Communication link failure: 1153 Got a packet bigger than 'max_allowed_packet' bytes (a3b0fb66)
(a3b0fb66) exception 'App\Exception\UpdateException' with message 'SQLSTATE[08S01]: Communication link failure: 1153 Got a packet bigger than 'max_allowed_packet' bytes' in :
Stack trace:
#0 (line): App\Database\DbUtils::directDbCore('INSERT INTO imp...', 'query', '')
#1 (line): App\Database\DbUtils::directDb('INSERT INTO imp...', 'query', '')
...
#17 {main}

Finally, this error also occurred every time the other two occurred, though I’m not sure of its significance:

[Mon, 09 Apr 18 03:34:08 -0400][ERROR]  FAILED TO DB LOG!: Error while sending QUERY packet. PID=30216
#0 [internal function]: {closure}(2, 'Error while sen...', '/var/www/html/a...', 954, Array)
#1 [internal function]: PDOStatement->execute()
#2 [internal function]: Phalcon\Db\Adapter\Pdo->executePrepared(Object(PDOStatement), Array, Array)
#3 [internal function]: Phalcon\Db\Adapter\Pdo->execute('INSERT INTO `ex...', Array, Array)
#4 [internal function]: Phalcon\Db\Adapter->insert('exceptions', Array, Array, Array)
...
#22 {main}