Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QueuedTracking does not handle mysql COMMIT failures #17511

Open
infomotions opened this issue Apr 29, 2021 · 3 comments
Open

QueuedTracking does not handle mysql COMMIT failures #17511

infomotions opened this issue Apr 29, 2021 · 3 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc.

Comments

@infomotions
Copy link

Expected Behavior

The queues would be processed without error.

Current Behavior

Intermittently the Queued tracking plugin throws errors while processing its queue. When this occurs, the processing call fails and the queues grow without being relieved. Sometimes, the call will eventually go through and things will continue to process normally, though backlogs can get very large and sometimes the error persists. Below is an example of the error.

[2021-04-29 11:09:51] piwik.ERROR: Uncaught exception: /path_to_matomo/core/Tracker/Db/Mysqli.php(432): Commit failed Commit failed #0 /path_to_matomo/plugins/QueuedTracking/Queue/Processor/Handler.php(109): Piwik\Tracker\Db\Mysqli->commit() #1 /path_to_matomo/plugins/QueuedTracking/Queue/Processor.php(208): Piwik\Plugins\QueuedTracking\Queue\Processor\Handler->commit() #2 /path_to_matomo/plugins/QueuedTracking/Queue/Processor.php(143): Piwik\Plugins\QueuedTracking\Queue\Processor->processRequestSets() #3 /path_to_matomo/plugins/QueuedTracking/Commands/Process.php(86): Piwik\Plugins\QueuedTracking\Queue\Processor->process() #4 /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Command/Command.php(257): Piwik\Plugins\QueuedTracking\Commands\Process->execute() #5 /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php(874): Symfony\Component\Console\Command\Command->run() #6 /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php(195): Symfony\Component\Console\Application->doRunCommand() #7 [internal function]: Symfony\Component\Console\Application->doRun() #8 /path_to_matomo/core/Console.php(130): call_user_func() #9 /path_to_matomo/core/Access.php(673): Piwik\Console->Piwik{closure}() #10 /path_to_matomo/core/Console.php(131): Piwik\Access::doAsSuperUser() #11 /path_to_matomo/core/Console.php(82): Piwik\Console->doRunImpl() #12 /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php(126): Piwik\Console->doRun() #13 /path_to_matomo/console(32): Symfony\Component\Console\Application->run() #14 {main} {"exception":"[object] (Piwik\Tracker\Db\DbException(code: 0): Commit failed at /path_to_matomo/core/Tracker/Db/Mysqli.php:432)","ignoreInScreenWriter":true} {"class":"Piwik\FrontController","request_id":901677}

[2021-04-29 11:09:51] piwik.DEBUG: Loaded plugins: CoreHome, IntranetMeasurable, Actions, Referrers, UserLanguage, DevicesDetection, Goals, Events, UserCountry, GeoIp2, VisitTime, VisitorInterest, Login, UsersManager, SitesManager, PrivacyManager, Contents, BulkTracking, Resolution, Heartbeat, Marketplace, UserId, PagePerformance, CustomDimensions, AbTesting, ActivityLog, DeviceDetectorCache, FormAnalytics, MarketingCampaignsReporting, MediaAnalytics, QueuedTracking, ReferrersManager, CorePluginsAdmin, CoreAdminHome, WebsiteMeasurable, Diagnostics, CoreVisualizations, Proxy, API, Widgetize, Transitions, LanguagesManager, Dashboard, MultiSites, SEO, VisitsSummary, VisitFrequency, RssWidget, Monolog, TwoFactorAuth, Installation, CoreUpdater, CoreConsole, ScheduledReports, UserCountryMap, Live, ImageGraph, Annotations, MobileMessaging, SegmentEditor, Insights, Morpheus, Intl, CustomJsTracker, DBStats, CustomAlerts, CustomiseTranslations, CustomReports, Funnels, InvalidateReports, LogViewer, MultiChannelConversionAttribution, RollUpReporting, SearchEngineKeywordsPerformance, SecurityInfo, TasksTimetable, TreemapVisualization, UsersFlow, WhiteLabel [] {"class":"Piwik\Plugin\Manager","request_id":901677}

[Piwik\Tracker\Db\DbException]
Commit failed

Exception trace:
() at /path_to_matomo/core/Tracker/Db/Mysqli.php:432
Piwik\Tracker\Db\Mysqli->commit() at /path_to_matomo/plugins/QueuedTracking/Queue/Processor/Handler.php:109
Piwik\Plugins\QueuedTracking\Queue\Processor\Handler->commit() at /path_to_matomo/plugins/QueuedTracking/Queue/Processor.php:208
Piwik\Plugins\QueuedTracking\Queue\Processor->processRequestSets() at /path_to_matomo/plugins/QueuedTracking/Queue/Processor.php:143
Piwik\Plugins\QueuedTracking\Queue\Processor->process() at /path_to_matomo/plugins/QueuedTracking/Commands/Process.php:86
Piwik\Plugins\QueuedTracking\Commands\Process->execute() at /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:257
Symfony\Component\Console\Command\Command->run() at /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php:874
Symfony\Component\Console\Application->doRunCommand() at /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php:195
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /path_to_matomo/core/Console.php:130
Piwik\Console->Piwik{closure}() at /path_to_matomo/core/Access.php:673
Piwik\Access::doAsSuperUser() at /path_to_matomo/core/Console.php:131
Piwik\Console->doRunImpl() at /path_to_matomo/core/Console.php:82
Piwik\Console->doRun() at /path_to_matomo/vendor/symfony/console/Symfony/Component/Console/Application.php:126
Symfony\Component\Console\Application->run() at /path_to_matomo/console:32

Context

This may occur at random times and requires manual intervention with the queues to recover by disabling the queues and manually clearing them.

Your Environment

Matomo 4.2.1
PHP Version: 8.0.3

@infomotions infomotions added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label Apr 29, 2021
@diosmosis
Copy link
Member

Hi @infomotions, sorry you're experiencing this and thanks for reporting the issue. It does seem like QueuedTracking doesn't handle failed COMMITs, I'll mark this as a bug.

@diosmosis diosmosis added Bug For errors / faults / flaws / inconsistencies etc. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels May 5, 2021
@diosmosis diosmosis changed the title Queued Tracking error when processing queue QueuedTracking does not handle mysql COMMIT failures May 5, 2021
@infomotions
Copy link
Author

Has there been any progress on this issue? It makes the plugin unusable as once it happens in a specific queue, it can grow until redis runs out of memory and deletes records. Is there no way to clear a specific "bad" item from a queue?

@tsteur
Copy link
Member

tsteur commented Aug 11, 2021

@infomotions to understand this issue better do you maybe any more information why the commit would fail?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Development

No branches or pull requests

4 participants