Segments are not invalidated when invalidating specific plugin #19176

Enhancement For Prioritization
@Starker3 opened this Issue on May 5th 2022 Contributor

When using the console command core:invalidate-report-data there is the option to only invalidate a specific plugin.
This is quite useful in certain circumstances (For example for CustomReports or other plugins). However, it seems that when executing this command with the --plugin option it skips the invalidation of the Segments for this plugin:

innocraft-staff-richard<a class='mention' href='https://github.com/matomo'>@matomo</a>:~/matomo4$ ./console core:invalidate-report-data --dates=2022-02-01,2022-02-05 --sites=1 --plugin=CustomReports
INFO [2022-05-05 01:00:41] 2066130  Invalidating day periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating week periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating month periods in 2022-02-01,2022-02-05 [segment = ]...
INFO [2022-05-05 01:00:41] 2066130  Invalidating year periods in 2022-02-01,2022-02-05 [segment = ]...
innocraft-staff-richard<a class='mention' href='https://github.com/matomo'>@matomo</a>:~/matomo4$ ./console core:archive --force-date-range=2022-02-01,2022-02-05 --force-idsites=1
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  INIT
INFO [2022-05-05 01:00:46] 2066199  Running Matomo 4.9.0 as Super User
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  NOTES
INFO [2022-05-05 01:00:46] 2066199  - Async process archiving supported, using CliMulti.
INFO [2022-05-05 01:00:46] 2066199  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-05 01:00:46] 2066199  - Archiving was last executed without error 19s ago.
INFO [2022-05-05 01:00:46] 2066199  - The following websites do not use the tracker: 
INFO [2022-05-05 01:00:46] 2066199  - Will process 1 websites (--force-idsites)
INFO [2022-05-05 01:00:46] 2066199  - Will process specified sites: 1
INFO [2022-05-05 01:00:46] 2066199  ---------------------------
INFO [2022-05-05 01:00:46] 2066199  START
INFO [2022-05-05 01:00:46] 2066199  Starting Matomo reports archiving...
INFO [2022-05-05 01:00:46] 2066199  Start processing archives for site 1.
INFO [2022-05-05 01:00:46] 2066199    Will invalidate archived reports for today in site ID = 1's timezone (2022-05-05 00:00:00).
INFO [2022-05-05 01:00:46] 2066199    Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-05-04 00:00:00).
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-05, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-04, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:46] 2066199  Archived website id 1, period = day, date = 2022-02-03, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.517s
INFO [2022-05-05 01:00:47] 2066199  Archived website id 1, period = day, date = 2022-02-02, segment = '', plugin = CustomReports, 8 visits found. Time elapsed: 0.769s
INFO [2022-05-05 01:00:47] 2066199  Archived website id 1, period = day, date = 2022-02-01, segment = '', plugin = CustomReports, 0 visits found. Time elapsed: 0.769s
INFO [2022-05-05 01:00:48] 2066199  Archived website id 1, period = month, date = 2022-02-01, segment = '', plugin = CustomReports, 216 visits found. Time elapsed: 0.458s
INFO [2022-05-05 01:00:48] 2066199  Archived website id 1, period = week, date = 2022-01-31, segment = '', plugin = CustomReports, 9 visits found. Time elapsed: 0.453s
INFO [2022-05-05 01:00:48] 2066199  Finished archiving for site 1, 7 API requests, Time elapsed: 2.561s [1 / 1 done]
INFO [2022-05-05 01:00:48] 2066199  Done archiving!
INFO [2022-05-05 01:00:48] 2066199  ---------------------------
INFO [2022-05-05 01:00:48] 2066199  SUMMARY
INFO [2022-05-05 01:00:48] 2066199  Processed 7 archives.
INFO [2022-05-05 01:00:48] 2066199  Total API requests: 7
INFO [2022-05-05 01:00:48] 2066199  done: 7 req, 2563 ms, no error
INFO [2022-05-05 01:00:48] 2066199  Time elapsed: 2.563s
INFO [2022-05-05 01:00:48] 2066199  ---------------------------

This then requires us to specify the segments to invalidate for the plugin which then works:

innocraft-staff-richard<a class='mention' href='https://github.com/matomo'>@matomo</a>:~/matomo4$ ./console core:invalidate-report-data --dates=2022-02-01,2022-02-05 --sites=1 --plugin=CustomReports --segment=5
INFO [2022-05-05 01:05:22] 2079165  Invalidating day periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating week periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating month periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
INFO [2022-05-05 01:05:22] 2079165  Invalidating year periods in 2022-02-01,2022-02-05 [segment = countryCode==nz]...
innocraft-staff-richard<a class='mention' href='https://github.com/matomo'>@matomo</a>:~/matomo4$ ./console core:archive --force-date-range=2022-02-01,2022-02-05 --force-idsites=1
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  INIT
INFO [2022-05-05 01:05:32] 2079472  Running Matomo 4.9.0 as Super User
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  NOTES
INFO [2022-05-05 01:05:32] 2079472  - Async process archiving supported, using CliMulti.
INFO [2022-05-05 01:05:32] 2079472  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-05 01:05:32] 2079472  - Archiving was last executed without error 4 min 42s ago.
INFO [2022-05-05 01:05:32] 2079472  - The following websites do not use the tracker: 
INFO [2022-05-05 01:05:32] 2079472  - Will process 1 websites (--force-idsites)
INFO [2022-05-05 01:05:32] 2079472  - Will process specified sites: 1
INFO [2022-05-05 01:05:32] 2079472  ---------------------------
INFO [2022-05-05 01:05:32] 2079472  START
INFO [2022-05-05 01:05:32] 2079472  Starting Matomo reports archiving...
INFO [2022-05-05 01:05:32] 2079472  Start processing archives for site 1.
INFO [2022-05-05 01:05:32] 2079472    Will invalidate archived reports for today in site ID = 1's timezone (2022-05-05 00:00:00).
INFO [2022-05-05 01:05:32] 2079472    Will invalidate archived reports for yesterday in site ID = 1's timezone (2022-05-04 00:00:00).
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-05, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-04, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-03, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.518s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-02, segment = 'countryCode==nz', plugin = CustomReports, 8 visits found. Time elapsed: 0.770s
INFO [2022-05-05 01:05:33] 2079472  Archived website id 1, period = day, date = 2022-02-01, segment = 'countryCode==nz', plugin = CustomReports, 0 visits found. Time elapsed: 0.770s
INFO [2022-05-05 01:05:34] 2079472  Archived website id 1, period = month, date = 2022-02-01, segment = 'countryCode==nz', plugin = CustomReports, 216 visits found. Time elapsed: 0.540s
INFO [2022-05-05 01:05:34] 2079472  Archived website id 1, period = week, date = 2022-01-31, segment = 'countryCode==nz', plugin = CustomReports, 8 visits found. Time elapsed: 0.459s
INFO [2022-05-05 01:05:35] 2079472  Finished archiving for site 1, 7 API requests, Time elapsed: 2.696s [1 / 1 done]
INFO [2022-05-05 01:05:35] 2079472  Done archiving!
INFO [2022-05-05 01:05:35] 2079472  ---------------------------
INFO [2022-05-05 01:05:35] 2079472  SUMMARY
INFO [2022-05-05 01:05:35] 2079472  Processed 7 archives.
INFO [2022-05-05 01:05:35] 2079472  Total API requests: 7
INFO [2022-05-05 01:05:35] 2079472  done: 7 req, 2698 ms, no error
INFO [2022-05-05 01:05:35] 2079472  Time elapsed: 2.698s
INFO [2022-05-05 01:05:35] 2079472  ---------------------------
INFO [2022-05-05 01:05:35] 2079472  SCHEDULED TASKS
INFO [2022-05-05 01:05:35] 2079472  Starting Scheduled tasks... 
INFO [2022-05-05 01:05:35] 2079472  done
INFO [2022-05-05 01:05:35] 2079472  ---------------------------

Invalidating the report without the --plugin options works as expected (It invalidates all segments)

@sgiehl commented on November 9th 2022 Member

Looking at the code it looks like segments are only invalidated if they are provided on command line. So even when not using the --plugin option it will not invalidate any segment.
Nevertheless the suggestion makes sense. Guess it would be good to add a new option like --include-all-segments or similar.

Home | Back
Powered by GitHub Issue Mirror
Privacy Policy