Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL >= 5.7.5 breaks Piwik <= 2.15.0 #9131

Closed
otheus opened this issue Oct 30, 2015 · 9 comments
Closed

MySQL >= 5.7.5 breaks Piwik <= 2.15.0 #9131

otheus opened this issue Oct 30, 2015 · 9 comments
Assignees
Labels
Bug For errors / faults / flaws / inconsistencies etc. duplicate For issues that already existed in our issue tracker and were reported previously.
Milestone

Comments

@otheus
Copy link

otheus commented Oct 30, 2015

A friendly write-up is here.

From the MySQL Documentation:

MySQL 5.7.5 and later enforces a maximum length on lock names of 64 characters. Previously, no limit was enforced

The typical string generated by Piwik for this purposes is 65 characters in length. It typically looks like this:

deletePreviousArchiveStatus.piwik_archive_numeric_2015_10.1267863

The get_lock() function is invoked from two functions within core/db.php. The two functions are getDbLock and releaseDbLock. For purposes of expediency on our systems, I wrapped the string used by these functions into a call to md5(), guaranteeing 32 characters at most, and almost ensuring uniqueness. I hope the devs come up with something better.

      if ($db->fetchOne($sql, array(md5($lockName))) == '1') {

These two functions are in turn invoked via acquireArchiveTableLock and releaseArchiveTableLock, both in DataAccess/Model.php.

@tsteur tsteur added the Bug For errors / faults / flaws / inconsistencies etc. label Oct 30, 2015
@tsteur tsteur added this to the 2.15.1 milestone Oct 30, 2015
@tsteur
Copy link
Member

tsteur commented Oct 30, 2015

Thx for the report!

@mattab
Copy link
Member

mattab commented Oct 30, 2015

We need to add Mysql 5.7 support in our Long Term Support release, added to 2.15.1: #6642

@otheus
Copy link
Author

otheus commented Nov 3, 2015

Presumably, the generated label is usable for debugging, especially in those edge cases where a lock blocks for a long time, then the server goes away, and some dba needs to clean it up. (At least, it might work on innodb tables.) My md5() workaround clobbers the utility of such code. That code is here: core/Model.php:284.

        $dbLockName = "deletePreviousArchiveStatus.$numericTable.$archiveId";

The problem is this: the code is completely MySQL-specific. Acquiring a "named" table lock is MySQL specific, (as is the need for table locks, in general), so to abstract from the db-engine, you'd need to push the name-generation down a level, into the Db-handler code itself. I'm not here today to preach the glorious benefits of using postgresql over mysql, so let's assume it will remain mysql-specific for the near future.

So while we're still talking MySQL-specific code, and since it's MySQL that forces us now to restrict this call to 64 characters, let's shorten it in a reasonable but mysql-specific way. Let me suggest: (1) use hex-encoded archiveID, (2) abbreviate the module prefix name.

The archiveID for that table type (see Db/Schema/Mysql.php) is an unsigned integer, or 4-bytes, or 8-hex chars. We explicate the length of the other fields, leaving up to 40 characters for the name of the table.

    // Ensure compat with 5.7.5+ : max 64-char get_lock() label
    $dbLockName = sprintf("%.14s.%.40s.%08x",
            "delPrvArcvStat", # 14 chars
            $numericTable,
            intval($archiveId)
    );

Is this enough to expedite progress, or should I generate an actual patch?

@otheus
Copy link
Author

otheus commented Nov 3, 2015

My PHP isn't that great, but maybe instead of relying on sprintf's precision, it makes more sense to do something like:

 substr($numericTable, -40, 40),

in order to make sure the lock identifies the most distinctive part of the table name.

@mattab
Copy link
Member

mattab commented Nov 24, 2015

Marked as duplicate of #6642 - edit: keeping this one open as well as it is more clear

@otheus
Copy link
Author

otheus commented Dec 21, 2015

@tsteur thank you for addressing this. Your patch does appear to solve the problem very thoroughly.

I don't want to sound presumptuous in addressing the following point on coding style: The fix in core/DataAccess/Model.php has a hard-coded constant of 23 without explanation whatsoever. I'm sure string interpolation in PHP is fast enough, but for clarity, might I suggest using sprintf with constants, ala #9131 (comment) ? Also, given the relative opacity of these MySQL archiveIDs, does it hurt to convert to hex?

@tsteur
Copy link
Member

tsteur commented Dec 21, 2015

@tsteur
Copy link
Member

tsteur commented Dec 21, 2015

and also see #9395

@otheus
Copy link
Author

otheus commented Dec 23, 2015

tyvm

@mattab mattab added the duplicate For issues that already existed in our issue tracker and were reported previously. label Jan 29, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. duplicate For issues that already existed in our issue tracker and were reported previously.
Projects
None yet
Development

No branches or pull requests

3 participants