Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Matomo 4 migration: Row size too large #16810

Closed
Findus23 opened this issue Nov 26, 2020 · 12 comments · Fixed by #16835
Closed

Matomo 4 migration: Row size too large #16810

Findus23 opened this issue Nov 26, 2020 · 12 comments · Fixed by #16835
Labels
Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone

Comments

@Findus23
Copy link
Member

reported by two people in https://forum.matomo.org/t/error-while-updating-to-4-0-0-b1-have-to-change-some-columns-to-text-or-blobs/39482

Critical Error during the update process:
/home/members/artisticactivism/sites/c4aa.org/web/matomo_stats/core/Updates/4.0.0-b1.php: Error trying to execute the migration 'ALTER TABLE matomo_log_visit ADD COLUMN visitor_seconds_since_first INT(11) UNSIGNED NULL, ADD COLUMN visitor_seconds_since_order INT(11) UNSIGNED NULL, ADD COLUMN visitor_seconds_since_last INT(11) UNSIGNED NULL, ADD COLUMN profilable TINYINT(1) NULL;'. The error was: SQLSTATE[42000]: Syntax error or access violation: 1118 Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs

@Findus23 Findus23 added the Regression Indicates a feature used to work in a certain way but it no longer does even though it should. label Nov 26, 2020
@Findus23 Findus23 added this to the 4.0.1 milestone Nov 26, 2020
@sgiehl
Copy link
Member

sgiehl commented Nov 26, 2020

Guess that might only happen in innodb strict mode and depends on configured row format and some other settings.
There's a quite good troubleshooting here: https://mariadb.com/kb/en/troubleshooting-row-size-too-large-errors-with-innodb/

@tsteur
Copy link
Member

tsteur commented Nov 26, 2020

I suppose this problem comes from either having many custom variables or custom dimensions.

In case of many custom dimensions converting some to TEXT might help:

ALTER TABLE matomo_log_visit MODIFY COLUMN custom_dimension_1 TEXT, MODIFY COLUMN custom_dimension_2 TEXT,MODIFY COLUMN custom_dimension_3 TEXT;

or custom variables:

ALTER TABLE matomo_log_visit MODIFY COLUMN custom_var_k1 TEXT, MODIFY COLUMN custom_var_v1 TEXT,MODIFY COLUMN custom_var_v2 TEXT;

Otherwise as @sgiehl mentioned the https://mariadb.com/kb/en/troubleshooting-row-size-too-large-errors-with-innodb/ link will help.

We can't really do anything here otherwise unfortunately. Can we close this issue?

@tsteur tsteur modified the milestones: 4.0.1, 4.0.2 Nov 26, 2020
@C0rn3j
Copy link

C0rn3j commented Nov 27, 2020

There was a problem installing the plugin CustomDimensions:

SQLSTATE[42000]: Syntax error or access violation: 1118 Row size too large. The maximum row size for the used table type, not counting BLOBs, is 8126. This includes storage overhead, check the manual. You have to change some columns to TEXT or BLOBs

Curiously, I have had this issue when upgrading 3.14.1 to 4.0.1 BEFORE the utf8mb4 conversion, and it disappeared right after.

Using mariadb 10.5.8.

@tsteur tsteur modified the milestones: 4.0.2, 4.0.3 Nov 29, 2020
@tsteur
Copy link
Member

tsteur commented Nov 29, 2020

So far what might help fix this issue is :

Option 1 (confirmed it works)

ALTER TABLE log_visit ROW_FORMAT=DYNAMIC;

Option 2 (when many custom dimensions or custom variables)

In case you have many custom dimensions converting some of them to TEXT might help (you can convert as many as needed):

ALTER TABLE matomo_log_visit MODIFY COLUMN custom_dimension_1 TEXT, MODIFY COLUMN custom_dimension_2 TEXT,MODIFY COLUMN custom_dimension_3 TEXT;

or if you have many custom variables (same here can convert as many as needed):

ALTER TABLE matomo_log_visit MODIFY COLUMN custom_var_k1 TEXT, MODIFY COLUMN custom_var_v1 TEXT,MODIFY COLUMN custom_var_v2 TEXT;

Option 3 (Experimental)

Alternatively it might help to disable innodb strict mode like. Not 100% sure this helps though or is a good idea. And likely the setting should be turned on again after the migration.

SET GLOBAL innodb_strict_mode = 0;

@tsteur
Copy link
Member

tsteur commented Nov 29, 2020

Summing roughly our default columns when marketing campaign reporting is installed I end up roughly 28+8+16+200+3+11+11+30+11+8+5+255+255+1+4000+20+10+40+20+100+100+100+3+11+100+18+10+11+255+3+2000+(255*10)+(255*6)+44= 11K characters so I suppose this roughly translates to 11K row page size when innodb maybe allows slightly less than 8K.

The easiest fix be to lower referer_url to 1000 which should be still plenty of characters for the referrer url

@tsteur
Copy link
Member

tsteur commented Nov 29, 2020

select sum(character_maximum_length) from COLUMNS where table_name like '%log_visit' and table_schema like '%api4_3%' returns in my case 12969 . This is with 15 custom dimensions though. With only 5 custom dimensions in scope visit it is only 10K.

@tsteur
Copy link
Member

tsteur commented Nov 29, 2020

To close this issue we need to create an FAQ suggesting the first 2 options as a workaround.

tsteur added a commit that referenced this issue Nov 29, 2020
refs #16810

Avoids running into row size exceeded in some cases.
tsteur added a commit that referenced this issue Nov 29, 2020
refs #16810

Avoids running into row size exceeded in some cases.
@S-n-d
Copy link

S-n-d commented Dec 1, 2020

I did all the things stated in the FAQ, so option 1 and option 2. Still get the same error. Any thoughts? I have 0 custom dimensions and my custom_var rows are all empty.

@sgiehl
Copy link
Member

sgiehl commented Dec 1, 2020

@S-n-d would you mind sharing the table information of your log_visit table? (you can simply use a query like DESCRIBE log_visit;)

@S-n-d
Copy link

S-n-d commented Dec 1, 2020

@sgiehl thanks for you reply. I've restored a file and DB backup (from yesterday) because I couldn't get into the dashboard anymore. Seems I've gotten myself in a heap of other issues now. I'll get back to this when I get my backup up and running.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants