Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(core dumped) when upgrade to 4.3.0 #17613

Closed
lds-lds opened this issue May 24, 2021 · 15 comments
Closed

(core dumped) when upgrade to 4.3.0 #17613

lds-lds opened this issue May 24, 2021 · 15 comments
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.

Comments

@lds-lds
Copy link

lds-lds commented May 24, 2021

hello

i got this error when i updated matomo to version 4.3.0 by running php /home/matomo/www/matomo/console core:update

  Executing ALTER TABLE `piwik_segment` ADD COLUMN `hash` CHAR(32) NULL AFTER `definition`;... Done. [1 / 151]
Erreur de segmentation (core dumped)

only one treatment appears on the 151 ...

I launched the command again and got this result:
Everything is already up to date.
did I corrupt my database ?

Your Environment

  • Matomo Version: 4.3.0
  • PHP Version:7.2.34-21
  • Server Operating System: ubuntu/xenial
  • Additionally installed plugins:
@lds-lds lds-lds added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label May 24, 2021
@diosmosis
Copy link
Member

Hi @lds-lds, that's strange that it would say it's already up to date since updating the internal version is only done after the update process is complete. If you'd like to re-run the updates, you can run the following SQL: UPDATE matomo_option SET option_value = '{previousVersion}' WHERE option_name = 'version_core'. Note: running updates twice will not result in extra changes or an error.

@lds-lds
Copy link
Author

lds-lds commented May 26, 2021

hello @diosmosis

when I execute your order I get Rows matched: 1 Changed: 1 Warnings: 0

Does that mean that the 151 operations of the maj were correctly executed?

MariaDB [piwik]> UPDATE piwik_option SET option_value = '4.2.1' WHERE option_name = 'version_core';
Query OK, 1 row affected (0.001 sec)
Rows matched: 1  Changed: 1  Warnings: 0

@diosmosis
Copy link
Member

@lds-lds Apologies, I should have mentioned, that SQL just makes matomo think its at an older version. So when you load Matomo or run the core:update command, it will run the updates from 4.2.1. You still need to load Matomo and go through the upgrade or run core:update.

@lds-lds
Copy link
Author

lds-lds commented May 27, 2021

ok I understood that in the meantime
my problem remains that console core: update returns a core dumped

I upgraded the version of php to 7.4 and the version of mariadb to 10.4 the problem remains the same

@diosmosis
Copy link
Member

@lds-lds we'll have to try and find out where the segfault is happening. Can you check the following for me?

  • does the brute_force_log table have a login column?
  • does the segment table have a hash column?

And just to be clear, you are upgrading from Matomo 4.2, correct?

@lds-lds
Copy link
Author

lds-lds commented May 28, 2021

@diosmosis
Yes i upgrade from Matomo 4.2

Here is mariadb describe command for piwik_brute_force_log and piwik_segment for matomo 4.2

MariaDB [piwik]> describe piwik_brute_force_log;
+--------------------+-------------+------+-----+---------+----------------+
| Field              | Type        | Null | Key | Default | Extra          |
+--------------------+-------------+------+-----+---------+----------------+
| id_brute_force_log | bigint(11)  | NO   | PRI | NULL    | auto_increment |
| ip_address         | varchar(60) | YES  | MUL | NULL    |                |
| attempted_at       | datetime    | NO   |     | NULL    |                |
+--------------------+-------------+------+-----+---------+----------------+

MariaDB [piwik]> describe piwik_segment;
+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| idsegment          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name               | varchar(255) | NO   |     | NULL    |                |
| definition         | mediumtext   | NO   |     | NULL    |                |
| login              | varchar(100) | NO   |     | NULL    |                |
| enable_all_users   | tinyint(4)   | NO   |     | 0       |                |
| enable_only_idsite | int(11)      | YES  |     | NULL    |                |
| auto_archive       | tinyint(4)   | NO   |     | 0       |                |
| ts_created         | timestamp    | YES  |     | NULL    |                |
| ts_last_edit       | timestamp    | YES  |     | NULL    |                |
| deleted            | tinyint(4)   | NO   |     | 0       |                |
+--------------------+--------------+------+-----+---------+----------------+

@lds-lds
Copy link
Author

lds-lds commented May 28, 2021

@diosmosis

And after upgrade from 4.2.1 to 4.3.1

+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| id_brute_force_log | bigint(11)   | NO   | PRI | NULL    | auto_increment |
| ip_address         | varchar(60)  | YES  | MUL | NULL    |                |
| attempted_at       | datetime     | NO   |     | NULL    |                |
| login              | varchar(100) | YES  |     | NULL    |                |
+--------------------+--------------+------+-----+---------+----------------+
MariaDB [piwik]> describe piwik_segment;
+--------------------+--------------+------+-----+---------+----------------+
| Field              | Type         | Null | Key | Default | Extra          |
+--------------------+--------------+------+-----+---------+----------------+
| idsegment          | int(11)      | NO   | PRI | NULL    | auto_increment |
| name               | varchar(255) | NO   |     | NULL    |                |
| definition         | mediumtext   | NO   |     | NULL    |                |
| hash               | char(32)     | YES  |     | NULL    |                |
| login              | varchar(100) | NO   |     | NULL    |                |
| enable_all_users   | tinyint(4)   | NO   |     | 0       |                |
| enable_only_idsite | int(11)      | YES  |     | NULL    |                |
| auto_archive       | tinyint(4)   | NO   |     | 0       |                |
| ts_created         | timestamp    | YES  |     | NULL    |                |
| ts_last_edit       | timestamp    | YES  |     | NULL    |                |
| deleted            | tinyint(4)   | NO   |     | 0       |                |
+--------------------+--------------+------+-----+---------+----------------+

@lds-lds
Copy link
Author

lds-lds commented May 28, 2021

it's very strange
I have enabled mysql logs
when I first run core: update then core dumped and I can see in the logs that the update mysql requests are not passed

if I UPDATE piwik_option SET option_value = '4.2.1' WHERE option_name = 'version_core';
and restart core dump then the update requests are well executed

@diosmosis
Copy link
Member

@lds-lds That's odd... it looks like the updates were successfully run, and the failure occurs sometime afterwards. Perhaps the output is buffered and does not have a chance to be printed out. I'm not sure where the issue could be. Can you check if there are any plugins that are now deactivated after the update? And would you be able to set version_core to 4.2.1 then do a dry run of the update and post the output here?

@lds-lds
Copy link
Author

lds-lds commented May 31, 2021

@diosmosis
Hello
i just updated again from vm matomo backups with mysql logs enabled

1: with core: update

there is a coredump on SELECT * FROM piwik_segment and mysql update operations did not pass

183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_log_conversion
183 Query SHOW TABLES LIKE 'piwik_%'
183 Query SHOW TABLES LIKE 'piwik_archive_numeric%'
183 Query SHOW TABLES LIKE 'piwik_archive_blob%'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'MatomoUpdateHistory'
183 Query UPDATE piwik_option SET option_value = '4.2.1,4.1.0,3.14.1,', autoload = '0' WHERE option_name = 'MatomoUpdateHistory'
183 Query ALTER TABLE piwik_segment ADD COLUMN hash CHAR(32) NULL AFTER definition
183 Query UPDATE piwik_option SET option_value = '4.3.0-b3', autoload = '1' WHERE option_name = 'version_core'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'lastTrackerCronRun'
183 Query SELECT option_name, option_value FROM piwik_option WHERE option_name LIKE '%report\_to\invalidate\%'
183 Query SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
183 Query SELECT idsite, main_url as url FROM piwik_site
183 Query SELECT idsite, url FROM piwik_site_url
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedUserAgentsGlobal'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedIpsGlobal'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.useAnonymizedIpForVisitEnrichment'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAddressMaskLength'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.doNotTrackEnabled'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAnonymizerEnabled'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.forceCookielessTracking'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeUserId'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeOrderId'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeReferrer'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_enable'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_schedule_lowest_interval'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_older_than'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_max_rows_per_query'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_schedule_lowest_interval'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_max_rows_per_query'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_auto_database_size_estimate'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_database_size_estimate'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_enable'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_older_than'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_basic_metrics'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_day_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_week_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_month_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_year_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_range_reports'
183 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_segment_reports'
183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_log_conversion
183 Query SHOW COLUMNS FROM piwik_log_link_visit_action
183 Query SHOW COLUMNS FROM piwik_log_visit
183 Query SHOW COLUMNS FROM piwik_log_conversion
183 Query SELECT * FROM piwik_segment

2: only via webinstaller

no errors all is good

      186 Query	SET NAMES 'utf8mb4'
      186 Query	SET sql_mode = "NO_AUTO_VALUE_ON_ZERO"
      186 Query	SELECT option_value, option_name FROM `piwik_option` WHERE autoload = 1
      186 Query	SELECT option_value FROM `piwik_option` WHERE option_name = 'usercountry.location_provider'
      186 Query	SHOW COLUMNS FROM piwik_log_visit
      186 Query	SHOW COLUMNS FROM piwik_log_link_visit_action
      186 Query	SHOW COLUMNS FROM piwik_log_conversion
      186 Query	SHOW TABLES LIKE 'piwik\_%'
      186 Query	SHOW TABLES LIKE 'piwik\_archive_numeric%'
      186 Query	SHOW TABLES LIKE 'piwik\_archive_blob%'
      186 Query	SELECT * FROM piwik_segment

210531 11:28:23 186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_log_conversion
186 Query SHOW TABLES LIKE 'piwik_%'
186 Query SHOW TABLES LIKE 'piwik_archive_numeric%'
186 Query SHOW TABLES LIKE 'piwik_archive_blob%'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'MatomoUpdateHistory'
186 Query ALTER TABLE piwik_segment ADD COLUMN hash CHAR(32) NULL AFTER definition
186 Query UPDATE piwik_option SET option_value = '4.3.0-b3', autoload = '1' WHERE option_name = 'version_core'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'lastTrackerCronRun'
186 Query SELECT option_name, option_value FROM piwik_option WHERE option_name LIKE '%report\_to\invalidate\%'
186 Query SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
186 Query SELECT idsite, main_url as url FROM piwik_site
186 Query SELECT idsite, url FROM piwik_site_url
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedUserAgentsGlobal'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'SitesManager_ExcludedIpsGlobal'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.useAnonymizedIpForVisitEnrichment'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAddressMaskLength'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.doNotTrackEnabled'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.ipAnonymizerEnabled'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.forceCookielessTracking'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeUserId'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeOrderId'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'PrivacyManager.anonymizeReferrer'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_enable'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_schedule_lowest_interval'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_older_than'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_max_rows_per_query'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_schedule_lowest_interval'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_logs_unused_actions_max_rows_per_query'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_auto_database_size_estimate'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'enable_database_size_estimate'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_enable'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_older_than'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_basic_metrics'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_day_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_week_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_month_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_year_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_range_reports'
186 Query SELECT option_value FROM piwik_option WHERE option_name = 'delete_reports_keep_segment_reports'
186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_log_conversion
186 Query SHOW COLUMNS FROM piwik_log_link_visit_action
186 Query SHOW COLUMNS FROM piwik_log_visit
186 Query SHOW COLUMNS FROM piwik_log_conversion
186 Query SELECT * FROM piwik_segment

      186 Query	UPDATE `piwik_segment` SET `hash` = 'xxxxxxxxxxxxxx' WHERE `idsegment` = '163'
        .............. etc etc .....
      186 Query	UPDATE `piwik_option` SET option_value = '4.3.0-b4', autoload = '1' WHERE option_name = 'version_core'
      186 Query	ALTER TABLE `piwik_brute_force_log` ADD COLUMN `login` VARCHAR(100) NULL
      186 Query	UPDATE `piwik_option` SET option_value = '4.3.0-rc2', autoload = '1' WHERE option_name = 'version_core'
      186 Query	UPDATE `piwik_option` SET option_value = '4.3.1', autoload = '1' WHERE option_name = 'version_core'
      186 Query	SELECT option_name, option_value FROM `piwik_option` WHERE option_name LIKE '%report\\_to\\_invalidate\\_%'
      186 Query	SHOW INDEX FROM piwik_log_visit WHERE Key_name = 'index_idsite_idvisitor'
      186 Query	SELECT idsite, main_url as url FROM piwik_site
      186 Query	SELECT idsite, url FROM piwik_site_url
      186 Query	SHOW COLUMNS FROM `piwik_log_visit`
      186 Query	SHOW COLUMNS FROM `piwik_log_link_visit_action`
      186 Query	SHOW COLUMNS FROM `piwik_log_conversion`
      186 Query	SHOW COLUMNS FROM `piwik_log_link_visit_action`
      186 Query	SHOW COLUMNS FROM `piwik_log_visit`
      186 Query	SHOW COLUMNS FROM `piwik_log_conversion`

@diosmosis
Copy link
Member

@lds-lds thanks for the extra information! My guess is the CLI php is either a different version than the web PHP version or they are configured differently. Perhaps the CLI php version is not allowed as much memory? Can you check the phpinfo() results of your CLI php vs. web php?

@lds-lds
Copy link
Author

lds-lds commented Jun 7, 2021

hello @diosmosis
memory limit in php.ini/cli is unlimited : memory_limit = -1

@lds-lds
Copy link
Author

lds-lds commented Jun 7, 2021

here is the diff

diff /etc/php/7.2/apache2/php.ini /etc/php/7.2/cli/php.ini

310c310
< disable_functions = pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wifcontinued,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_get_handler,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,pcntl_async_signals,
---
> disable_functions =
371c371
< expose_php = Off
---
> expose_php = On
380c380
< max_execution_time = 600
---
> max_execution_time = 30
401c401
< memory_limit = 2048M
---
> memory_limit = -1

@lds-lds
Copy link
Author

lds-lds commented Jun 7, 2021

strange if i run it twice in a row the update works
1: php /home/matomo/www/matomo/console core:update
Erreur de segmentation
2: php /home/matomo/www/matomo/console core:update
all OK

O-o

I will launch the update from the browser, since this way I have no problems

@diosmosis
Copy link
Member

Hi @lds-lds, ok good to know you have a workaround! I'll close this ticket for now, but if you notice any other cli commands segfaulting feel free to open this ticket again. If you want to try and figure out exactly what's causing the problem, we can, but it will take a while to isolate the cause.

strange if i run it twice in a row the update works

Note: this makes sense if it's running the updates successfully and then erroring later. The version is updated in the database, so the next time you run it, it won't try to run any updates, since the database is already up-to-date.

@mattab mattab added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Jul 27, 2021
@tsteur tsteur removed the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Sep 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.
Projects
None yet
Development

No branches or pull requests

4 participants