@lds-lds opened this Issue on May 24th 2021

hello

i got this error when i updated matomo to version 4.3.0 by running php /home/matomo/www/matomo/console core:update

  Executing ALTER TABLE `piwik_segment` ADD COLUMN `hash` CHAR(32) NULL AFTER `definition`;... Done. [1 / 151]
Erreur de segmentation (core dumped)

only one treatment appears on the 151 ...

I launched the command again and got this result:
Everything is already up to date.
did I corrupt my database ?

Your Environment

  • Matomo Version: 4.3.0
  • PHP Version:7.2.34-21
  • Server Operating System: ubuntu/xenial
  • Additionally installed plugins:
@diosmosis commented on May 24th 2021 Member

Hi @lds-lds, that's strange that it would say it's already up to date since updating the internal version is only done after the update process is complete. If you'd like to re-run the updates, you can run the following SQL: UPDATE matomo_option SET option_value = '{previousVersion}' WHERE option_name = 'version_core'. Note: running updates twice will not result in extra changes or an error.

@lds-lds commented on May 26th 2021

hello @diosmosis

when I execute your order I get Rows matched: 1 Changed: 1 Warnings: 0

Does that mean that the 151 operations of the maj were correctly executed?

MariaDB [piwik]> UPDATE piwik_option SET option_value = '4.2.1' WHERE option_name = 'version_core';
Query OK, 1 row affected (0.001 sec)
Rows matched: 1  Changed: 1  Warnings: 0
@diosmosis commented on May 26th 2021 Member

@lds-lds Apologies, I should have mentioned, that SQL just makes matomo think its at an older version. So when you load Matomo or run the core:update command, it will run the updates from 4.2.1. You still need to load Matomo and go through the upgrade or run core:update.

@lds-lds commented on May 27th 2021

ok I understood that in the meantime
my problem remains that console core: update returns a core dumped

I upgraded the version of php to 7.4 and the version of mariadb to 10.4 the problem remains the same

@diosmosis commented on May 27th 2021 Member

@lds-lds we'll have to try and find out where the segfault is happening. Can you check the following for me?

  • does the brute_force_log table have a login column?
  • does the segment table have a hash column?

And just to be clear, you are upgrading from Matomo 4.2, correct?

@lds-lds commented on May 28th 2021

@diosmosis
Yes i upgrade from Matomo 4.2

Here is mariadb describe command for piwik_brute_force_log and piwik_segment for matomo 4.2

MariaDB [piwik]> describe piwik_brute_force_log;
+--------------------+-------------+------+-----+---------+----------------+
| Field              | Type        | Null | Key | Default | Extra          |
+--------------------+-------------+------+-----+---------+----------------+
| id_brute_force_log | bigint(11)  | NO   | PRI | NULL    | auto_increment |
| ip_address         | varchar(60) | YES  | MUL | NULL    |                |
| attempted_at       | datetime    | NO   |     | NULL    |                |
+--------------------+-------------+------+-----+---------+----------------+
MariaDB [piwik]> describe piwik_segment;
+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| idsegment          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name               | varchar(255) | NO   |     | NULL    |                |
| definition         | mediumtext   | NO   |     | NULL    |                |
| login              | varchar(100) | NO   |     | NULL    |                |
| enable_all_users   | tinyint(4)   | NO   |     | 0       |                |
| enable_only_idsite | int(11)      | YES  |     | NULL    |                |
| auto_archive       | tinyint(4)   | NO   |     | 0       |                |
| ts_created         | timestamp    | YES  |     | NULL    |                |
| ts_last_edit       | timestamp    | YES  |     | NULL    |                |
| deleted            | tinyint(4)   | NO   |     | 0       |                |
+--------------------+--------------+------+-----+---------+----------------+
@lds-lds commented on May 28th 2021

@diosmosis

And after upgrade from 4.2.1 to 4.3.1

+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| id_brute_force_log | bigint(11)   | NO   | PRI | NULL    | auto_increment |
| ip_address         | varchar(60)  | YES  | MUL | NULL    |                |
| attempted_at       | datetime     | NO   |     | NULL    |                |
| login              | varchar(100) | YES  |     | NULL    |                |
+--------------------+--------------+------+-----+---------+----------------+
MariaDB [piwik]> describe piwik_segment;
+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| idsegment          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name               | varchar(255) | NO   |     | NULL    |                |
| definition         | mediumtext   | NO   |     | NULL    |                |
| hash               | char(32)     | YES  |     | NULL    |                |
| login              | varchar(100) | NO   |     | NULL    |                |
| enable_all_users   | tinyint(4)   | NO   |     | 0       |                |
| enable_only_idsite | int(11)      | YES  |     | NULL    |                |
| auto_archive       | tinyint(4)   | NO   |     | 0       |                |
| ts_created         | timestamp    | YES  |     | NULL    |                |
| ts_last_edit       | timestamp    | YES  |     | NULL    |                |
| deleted            | tinyint(4)   | NO   |     | 0       |                |
+--------------------+--------------+------+-----+---------+----------------+
@lds-lds commented on May 28th 2021

it's very strange
I have enabled mysql logs
when I first run core: update then core dumped and I can see in the logs that the update mysql requests are not passed

if I UPDATE piwik_option SET option_value = '4.2.1' WHERE option_name = 'version_core';
and restart core dump then the update requests are well executed

@diosmosis commented on May 30th 2021 Member

@lds-lds That's odd... it looks like the updates were successfully run, and the failure occurs sometime afterwards. Perhaps the output is buffered and does not have a chance to be printed out. I'm not sure where the issue could be. Can you check if there are any plugins that are now deactivated after the update? And would you be able to set version_core to 4.2.1 then do a dry run of the update and post the output here?

@lds-lds commented on May 31st 2021

@diosmosis
Hello
i just updated again from vm matomo backups with mysql logs enabled

1: with core: update

there is a coredump on SELECT * FROM piwik_segment and mysql update operations did not pass

183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_logconversion
183 Query SHOW TABLES LIKE 'piwik\
%'
183 Query SHOW TABLES LIKE 'piwik_archive_numeric%'
183 Query SHOW TABLES LIKE 'piwik_archive_blob%'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'MatomoUpdateHistory'
183 Query UPDATE piwik_option SET option_value = '4.2.1,4.1.0,3.14.1,', autoload = '0' WHERE option_name = 'MatomoUpdateHistory'
183 Query ALTER TABLE piwik_segment ADD COLUMN hash CHAR(32) NULL AFTER definition
183 Query UPDATE piwik_option SET option_value = '4.3.0-b3', autoload = '1' WHERE option_name = 'version_core'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'lastTrackerCronRun'
183 Query SELECT option_name, option_value FROM piwik_option WHERE optionname LIKE '%report\_to\_invalidate\%'
183 Query SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
183 Query SELECT idsite, main_url as url FROM piwik_site
183 Query SELECT idsite, url FROM piwik_site_url
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedUserAgentsGlobal'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedIpsGlobal'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.useAnonymizedIpForVisitEnrichment'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAddressMaskLength'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.doNotTrackEnabled'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAnonymizerEnabled'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.forceCookielessTracking'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeUserId'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeOrderId'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeReferrer'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_enable'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_schedule_lowest_interval'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_older_than'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_max_rows_per_query'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_schedule_lowest_interval'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_max_rows_per_query'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_auto_database_size_estimate'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_database_size_estimate'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_enable'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_older_than'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_basic_metrics'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_day_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_week_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_month_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_year_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_range_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_segment_reports'
183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_log_conversion
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_conversion
183 Query SELECT * FROM piwik_segment

2: only via webinstaller

no errors all is good

      186 Query SET NAMES 'utf8mb4'
      186 Query SET sql_mode = "NO_AUTO_VALUE_ON_ZERO"
      186 Query SELECT option_value, option_name FROM `piwik_option` WHERE autoload = 1
      186 Query SELECT option_value FROM `piwik_option` WHERE option_name = 'usercountry.location_provider'
      186 Query SHOW COLUMNS FROM piwik_log_visit
      186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
      186 Query SHOW COLUMNS FROM piwik_log_conversion
      186 Query SHOW TABLES LIKE 'piwik\_%'
      186 Query SHOW TABLES LIKE 'piwik\_archive_numeric%'
      186 Query SHOW TABLES LIKE 'piwik\_archive_blob%'
      186 Query SELECT * FROM piwik_segment

210531 11:28:23 186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_logconversion
186 Query SHOW TABLES LIKE 'piwik\
%'
186 Query SHOW TABLES LIKE 'piwik_archive_numeric%'
186 Query SHOW TABLES LIKE 'piwik_archive_blob%'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'MatomoUpdateHistory'
186 Query ALTER TABLE piwik_segment ADD COLUMN hash CHAR(32) NULL AFTER definition
186 Query UPDATE piwik_option SET option_value = '4.3.0-b3', autoload = '1' WHERE option_name = 'version_core'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'lastTrackerCronRun'
186 Query SELECT option_name, option_value FROM piwik_option WHERE optionname LIKE '%report\_to\_invalidate\%'
186 Query SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
186 Query SELECT idsite, main_url as url FROM piwik_site
186 Query SELECT idsite, url FROM piwik_site_url
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedUserAgentsGlobal'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedIpsGlobal'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.useAnonymizedIpForVisitEnrichment'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAddressMaskLength'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.doNotTrackEnabled'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAnonymizerEnabled'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.forceCookielessTracking'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeUserId'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeOrderId'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeReferrer'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_enable'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_schedule_lowest_interval'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_older_than'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_max_rows_per_query'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_schedule_lowest_interval'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_max_rows_per_query'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_auto_database_size_estimate'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_database_size_estimate'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_enable'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_older_than'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_basic_metrics'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_day_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_week_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_month_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_year_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_range_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_segment_reports'
186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_log_conversion
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_conversion
186 Query SELECT * FROM piwik_segment

      186 Query UPDATE `piwik_segment` SET `hash` = 'xxxxxxxxxxxxxx' WHERE `idsegment` = '163'
        .............. etc etc .....
      186 Query UPDATE `piwik_option` SET option_value = '4.3.0-b4', autoload = '1' WHERE option_name = 'version_core'
      186 Query ALTER TABLE `piwik_brute_force_log` ADD COLUMN `login` VARCHAR(100) NULL
      186 Query UPDATE `piwik_option` SET option_value = '4.3.0-rc2', autoload = '1' WHERE option_name = 'version_core'
      186 Query UPDATE `piwik_option` SET option_value = '4.3.1', autoload = '1' WHERE option_name = 'version_core'
      186 Query SELECT option_name, option_value FROM `piwik_option` WHERE option_name LIKE '%report\\_to\\_invalidate\\_%'
      186 Query SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
      186 Query SELECT idsite, main_url as url FROM piwik_site
      186 Query SELECT idsite, url FROM piwik_site_url
      186 Query SHOW COLUMNS FROM `piwik_log_visit`
      186 Query SHOW COLUMNS FROM `piwik_log_link_visit_action`
      186 Query SHOW COLUMNS FROM `piwik_log_conversion`
      186 Query SHOW COLUMNS FROM `piwik_log_link_visit_action`
      186 Query SHOW COLUMNS FROM `piwik_log_visit`
      186 Query SHOW COLUMNS FROM `piwik_log_conversion`
@diosmosis commented on May 31st 2021 Member

@lds-lds thanks for the extra information! My guess is the CLI php is either a different version than the web PHP version or they are configured differently. Perhaps the CLI php version is not allowed as much memory? Can you check the phpinfo() results of your CLI php vs. web php?

@lds-lds commented on June 7th 2021

hello @diosmosis
memory limit in php.ini/cli is unlimited : memory_limit = -1

@lds-lds commented on June 7th 2021

here is the diff

diff /etc/php/7.2/apache2/php.ini /etc/php/7.2/cli/php.ini

310c310
< disable_functions = pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals,
---
> disable_functions =
371c371
< expose_php = Off
---
> expose_php = On
380c380
< max_execution_time = 600
---
> max_execution_time = 30
401c401
< memory_limit = 2048M
---
> memory_limit = -1
@lds-lds commented on June 7th 2021

strange if i run it twice in a row the update works
1: php /home/matomo/www/matomo/console core:update
Erreur de segmentation
2: php /home/matomo/www/matomo/console core:update
all OK

O-o

I will launch the update from the browser, since this way I have no problems

@diosmosis commented on June 7th 2021 Member

Hi @lds-lds, ok good to know you have a workaround! I'll close this ticket for now, but if you notice any other cli commands segfaulting feel free to open this ticket again. If you want to try and figure out exactly what's causing the problem, we can, but it will take a while to isolate the cause.

strange if i run it twice in a row the update works

Note: this makes sense if it's running the updates successfully and then erroring later. The version is updated in the database, so the next time you run it, it won't try to run any updates, since the database is already up-to-date.

This Issue was closed on June 7th 2021
Powered by GitHub Issue Mirror