New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve delete-logs-data memory usage #20038
Comments
May be linked to any changes happened which cause also #19695 |
Hi @grzchr15, thanks for reporting this. Issue 19695 occurs when archiving data so it's probably not related. Deleting log data can use quite a bit of memory. The code that ran out of memory in the fatal error above was building a database query with multiple visitor ids. It's possible that this area of code could be made more memory efficient. Does decreasing the |
retried it with --100 does not change anything |
Thanks for trying that @grzchr15. I'll allocate this issue for prioritization. Increasing the allowed PHP memory or breaking up the date ranges are probably the best options to get this to run for now. |
1.) it would be helpfull to have a way to calculate the needed memory for that. |
Expected Behavior
Delete logs data
Current Behavior
Fatal error: Allowed memory size of 1677721600 bytes exhausted (tried to allocate 20480 bytes) in /var/www/clients/client2/web282/web/piwik/plugins/PrivacyManager/Model/DataSubjects.php on line 497
Possible Solution
Steps to Reproduce (for Bugs)
1../console core:delete-logs-data --dates=2011-01-01,2017-12-31 --limit 1000 in /var/www/somepath/web/piwik/core/DataTable/Manager.php
2.PHP Fatal error: Allowed memory size of 1677721600 bytes exhausted (tried to allocate 20480 bytes) in /var/www/clients/client2/web282/web/piwik/plugins/PrivacyManager/Model/DataSubjects.php on line 497
Context
Your Environment
Matomo Version: ./console --version
Matomo version 4.12.3
PHP Version:
PHP 7.4.33 (cli) (built: Oct 31 2022 10:36:05) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
with Zend OPcache v7.4.33, Copyright (c), by Zend Technologies
Server Operating System: Redhat 9
Additionally installed plugins:
./console plugin:list
+-----------------------------------+-------------------+---------------+
| Plugin | Core or optional? | Status |
+-----------------------------------+-------------------+---------------+
| Diagnostics | Core | Activated |
| Resolution | Core | Activated |
| ScheduledReports | Core | Activated |
| UserCountryMap | Core | Activated |
| Live | Core | Activated |
| PrivacyManager | Core | Activated |
| ImageGraph | Core | Activated |
| Annotations | Core | Activated |
| MobileMessaging | Core | Activated |
| Login | Core | Activated |
| SegmentEditor | Core | Activated |
| Insights | Core | Activated |
| Morpheus | Core | Activated |
| Contents | Core | Activated |
| BulkTracking | Core | Activated |
| DevicePlugins | Core | Activated |
| CoreUpdater | Core | Activated |
| Heartbeat | Core | Activated |
| Marketplace | Core | Activated |
| ProfessionalServices | Core | Activated |
| UserId | Core | Activated |
| CustomDimensions | Core | Activated |
| DBStats | Core | Activated |
| MobileAppMeasurable | Core | Activated |
| IntranetMeasurable | Core | Activated |
| TwoFactorAuth | Core | Activated |
| GeoIp2 | Core | Activated |
| TagManager | Core | Activated |
| Tour | Core | Activated |
| CustomJsTracker | Core | Activated |
| PagePerformance | Core | Activated |
| CoreConsole | Core | Activated |
| Overlay | Core | Activated |
| Feedback | Core | Activated |
| LanguagesManager | Core | Activated |
| CoreAdminHome | Core | Activated |
| UsersManager | Core | Activated |
| SitesManager | Core | Activated |
| Installation | Core | Activated |
| Monolog | Core | Activated |
| Intl | Core | Activated |
| CorePluginsAdmin | Core | Activated |
| CoreHome | Core | Activated |
| RssWidget | Core | Activated |
| CoreVisualizations | Core | Activated |
| Proxy | Core | Activated |
| API | Core | Activated |
| Widgetize | Core | Activated |
| Transitions | Core | Activated |
| WebsiteMeasurable | Core | Activated |
| Actions | Core | Activated |
| Ecommerce | Core | Activated |
| VisitorInterest | Core | Activated |
| VisitTime | Core | Activated |
| VisitFrequency | Core | Activated |
| VisitsSummary | Core | Activated |
| Dashboard | Core | Activated |
| UserCountry | Core | Activated |
| Events | Core | Activated |
| SEO | Core | Activated |
| CoreVue | Core | Activated |
| Goals | Core | Activated |
| MultiSites | Core | Activated |
| Referrers | Core | Activated |
| UserLanguage | Core | Activated |
| DevicesDetection | Core | Activated |
| TreemapVisualization | Optional | Activated |
| UsersFlow | Optional | Activated |
| Migration | Optional | Activated |
| IPReports | Optional | Activated |
| CustomVariables | Optional | Activated |
| WooCommerceAnalytics | Optional | Not activated |
| LoginSaml | Optional | Activated |
| Funnels | Optional | Activated |
| MultiChannelConversionAttribution | Optional | Activated |
| RollUpReporting | Optional | Activated |
| LiveTab | Optional | Activated |
| FormAnalytics | Optional | Activated |
| ActivityLog | Optional | Activated |
| MediaAnalytics | Optional | Activated |
| WhiteLabel | Optional | Activated |
| HeatmapSessionRecording | Optional | Activated |
| SecurityInfo | Optional | Activated |
| CustomReports | Optional | Activated |
| SearchEngineKeywordsPerformance | Optional | Activated |
| Bandwidth | Optional | Not activated |
| LogViewer | Optional | Activated |
| Provider | Optional | Activated |
| AbTesting | Optional | Activated |
+-----------------------------------+-------------------+---------------+
The text was updated successfully, but these errors were encountered: