Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segments are not invalidated when invalidating specific plugin #19176

Open
Starker3 opened this issue May 5, 2022 · 1 comment
Open

Segments are not invalidated when invalidating specific plugin #19176

Starker3 opened this issue May 5, 2022 · 1 comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone
For Prioritization

Comments

@Starker3
Copy link
Contributor

Starker3 commented May 5, 2022

When using the console command core:invalidate-report-data there is the option to only invalidate a specific plugin.
This is quite useful in certain circumstances (For example for CustomReports or other plugins). However, it seems that when executing this command with the --plugin option it skips the invalidation of the Segments for this plugin:

innocraft-staff-richard@matomo:~/matomo4$ ./console core:invalidate-report-data --dates=2022-02-01,2022-02-05 --sites=1 --plugin=CustomReports
INFO [2022-05-05 01:00:41] 2066130  Invalidating day periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating week periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating month periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating year periods in 2022-02-01,2022-02-05 [segment = ]...
innocraft-staff-richard@matomo:~/matomo4$ ./console core:archive --force-date-range=2022-02-01,2022-02-05 --force-idsites=1
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  INIT
INFO [2022-05-05 01:00:46] 2066199  Running Matomo 4.9.0 as Super User
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  NOTES
INFO [2022-05-05 01:00:46] 2066199  - Async process archiving supported, using CliMulti.
INFO [2022-05-05 01:00:46] 2066199  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-05 01:00:46] 2066199  - Archiving was last executed without error 19s ago.
INFO [2022-05-05 01:00:46] 2066199  - The following websites do not use the tracker: 
INFO [2022-05-05 01:00:46] 2066199  - Will process 1 websites (--force-idsites)
INFO [2022-05-05 01:00:46] 2066199  - Will process specified sites: 1
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  START
INFO [2022-05-05 01:00:46] 2066199  Starting Matomo reports archiving...
INFO [2022-05-05 01:00:46] 2066199  Start processing archives for site 1.
INFO [2022-05-05 01:00:46] 2066199    Will invalidate archived reports for today in site ID = 1's timezone (2022-05-05 00:00:00).
INFO [2022-05-05 01:00:46] 2066199    Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-05-04 00:00:00).
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-05, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-04, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-03, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:47] 2066199  Archived website id 1, period = day, date = 2022-02-02, segment = '', plugin = CustomReports, 8 visits found. Time elapsed: 0.769s
INFO [2022-05-05 01:00:47] 2066199  Archived website id 1, period = day, date = 2022-02-01, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.769s
INFO [2022-05-05 01:00:48] 2066199  Archived website id 1, period = month, date = 2022-02-01, segment = '', plugin = CustomReports, 216 visits found. Time elapsed: 0.458s
INFO [2022-05-05 01:00:48] 2066199  Archived website id 1, period = week, date = 2022-01-31, segment = '', plugin = CustomReports, 9 visits found. Time elapsed: 0.453s
INFO [2022-05-05 01:00:48] 2066199  Finished archiving for site 1, 7 API requests, Time elapsed: 2.561s [1 / 1 done]
INFO [2022-05-05 01:00:48] 2066199  Done archiving!
INFO [2022-05-05 01:00:48] 2066199  ---------------------------
INFO [2022-05-05 01:00:48] 2066199  SUMMARY
INFO [2022-05-05 01:00:48] 2066199  Processed 7 archives.
INFO [2022-05-05 01:00:48] 2066199  Total API requests: 7
INFO [2022-05-05 01:00:48] 2066199  done: 7 req, 2563 ms, no error
INFO [2022-05-05 01:00:48] 2066199  Time elapsed: 2.563s
INFO [2022-05-05 01:00:48] 2066199  ---------------------------

This then requires us to specify the segments to invalidate for the plugin which then works:

innocraft-staff-richard@matomo:~/matomo4$ ./console core:invalidate-report-data --dates=2022-02-01,2022-02-05 --sites=1 --plugin=CustomReports --segment=5
INFO [2022-05-05 01:05:22] 2079165  Invalidating day periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating week periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating month periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating year periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
innocraft-staff-richard@matomo:~/matomo4$ ./console core:archive --force-date-range=2022-02-01,2022-02-05 --force-idsites=1
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  INIT
INFO [2022-05-05 01:05:32] 2079472  Running Matomo 4.9.0 as Super User
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  NOTES
INFO [2022-05-05 01:05:32] 2079472  - Async process archiving supported, using CliMulti.
INFO [2022-05-05 01:05:32] 2079472  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-05 01:05:32] 2079472  - Archiving was last executed without error 4 min 42s ago.
INFO [2022-05-05 01:05:32] 2079472  - The following websites do not use the tracker: 
INFO [2022-05-05 01:05:32] 2079472  - Will process 1 websites (--force-idsites)
INFO [2022-05-05 01:05:32] 2079472  - Will process specified sites: 1
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  START
INFO [2022-05-05 01:05:32] 2079472  Starting Matomo reports archiving...
INFO [2022-05-05 01:05:32] 2079472  Start processing archives for site 1.
INFO [2022-05-05 01:05:32] 2079472    Will invalidate archived reports for today in site ID = 1's timezone (2022-05-05 00:00:00).
INFO [2022-05-05 01:05:32] 2079472    Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-05-04 00:00:00).
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-05, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-04, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-03, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-02, segment = 'countryCode==nz', plugin = CustomReports, 8 visits found. Time elapsed: 0.770s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-01, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.770s
INFO [2022-05-05 01:05:34] 2079472  Archived website id 1, period = month, date = 2022-02-01, segment = 'countryCode==nz', plugin = CustomReports, 216 visits found. Time elapsed: 0.540s
INFO [2022-05-05 01:05:34] 2079472  Archived website id 1, period = week, date = 2022-01-31, segment = 'countryCode==nz', plugin = CustomReports, 8 visits found. Time elapsed: 0.459s
INFO [2022-05-05 01:05:35] 2079472  Finished archiving for site 1, 7 API requests, Time elapsed: 2.696s [1 / 1 done]
INFO [2022-05-05 01:05:35] 2079472  Done archiving!
INFO [2022-05-05 01:05:35] 2079472  ---------------------------
INFO [2022-05-05 01:05:35] 2079472  SUMMARY
INFO [2022-05-05 01:05:35] 2079472  Processed 7 archives.
INFO [2022-05-05 01:05:35] 2079472  Total API requests: 7
INFO [2022-05-05 01:05:35] 2079472  done: 7 req, 2698 ms, no error
INFO [2022-05-05 01:05:35] 2079472  Time elapsed: 2.698s
INFO [2022-05-05 01:05:35] 2079472  ---------------------------
INFO [2022-05-05 01:05:35] 2079472  SCHEDULED TASKS
INFO [2022-05-05 01:05:35] 2079472  Starting Scheduled tasks... 
INFO [2022-05-05 01:05:35] 2079472  done
INFO [2022-05-05 01:05:35] 2079472  ---------------------------

Invalidating the report without the --plugin options works as expected (It invalidates all segments)
@Starker3 Starker3 added the Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. label May 5, 2022
@sgiehl
Copy link
Member

sgiehl commented Nov 9, 2022

Looking at the code it looks like segments are only invalidated if they are provided on command line. So even when not using the --plugin option it will not invalidate any segment.
Nevertheless the suggestion makes sense. Guess it would be good to add a new option like --include-all-segments or similar.

@sgiehl sgiehl added Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. and removed Potential Bug Something that might be a bug, but needs validation and confirmation it can be reproduced. labels Nov 9, 2022
@sgiehl sgiehl added this to the For Prioritization milestone Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Milestone
For Prioritization
Development

No branches or pull requests
2 participants
@sgiehl @Starker3