Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Matomo v4.0.2 cron core:archive --force-periods broken #16832

Closed
kochtopf opened this issue Nov 28, 2020 · 4 comments
Closed

Matomo v4.0.2 cron core:archive --force-periods broken #16832

kochtopf opened this issue Nov 28, 2020 · 4 comments
Assignees
@diosmosis
Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Milestone
4.0.x

Comments

@kochtopf
Copy link

'sudo -u www-data /usr/bin/php /var/www/piwik/console core:archive --force-periods=day' does not work as in v3.14.
An archive with --force-periods does run OK exactly ONE time after a full core:archive (without additional options). To reproduce:

  1. run core:archive -> daily, weekly, yearly reports are recreated OK (does some API requests)
  2. let expire your 'Archive reports at most every X seconds = 150 seconds' period then run core:archive --force-periods=day -> this one runs OK (again does some API and new reports)
  3. wait another 150 seconds and do --force periods=day again -> Total API requests: 0. After that subsequent --force-periods=day do not do any reports -> API always 0. To reset this situation I had to go to 1. (create daily, weekly, yearly).
    Is this intentional or an undesired feature???

INFO [2020-11-28 14:14:14] 1315 ---------------------------
INFO [2020-11-28 14:14:14] 1315 INIT
INFO [2020-11-28 14:14:14] 1315 Running Matomo 4.0.2 as Super User
INFO [2020-11-28 14:14:14] 1315 ---------------------------
INFO [2020-11-28 14:14:14] 1315 NOTES
INFO [2020-11-28 14:14:15] 1315 - Async process archiving supported, using CliMulti.
INFO [2020-11-28 14:14:15] 1315 - Reports for today will be processed at most every 150 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-11-28 14:14:15] 1315 - Archiving was last executed without error 7 min 16s ago
INFO [2020-11-28 14:14:15] 1315 - Will only process the following periods: day (--force-periods)
INFO [2020-11-28 14:14:15] 1315 ---------------------------
INFO [2020-11-28 14:14:15] 1315 START
INFO [2020-11-28 14:14:15] 1315 Starting Matomo reports archiving...
INFO [2020-11-28 14:14:15] 1315 Start processing archives for site 1.
INFO [2020-11-28 14:14:15] 1315 Finished archiving for site 1, 0 API requests, Time elapsed: 0.019s [1 / 4 done]
INFO [2020-11-28 14:14:15] 1315 Start processing archives for site 2.
INFO [2020-11-28 14:14:15] 1315 Finished archiving for site 2, 0 API requests, Time elapsed: 0.081s [2 / 4 done]
INFO [2020-11-28 14:14:15] 1315 Start processing archives for site 3.
INFO [2020-11-28 14:14:15] 1315 Finished archiving for site 3, 0 API requests, Time elapsed: 0.024s [3 / 4 done]
INFO [2020-11-28 14:14:15] 1315 Start processing archives for site 4.
INFO [2020-11-28 14:14:15] 1315 Finished archiving for site 4, 0 API requests, Time elapsed: 0.018s [4 / 4 done]
INFO [2020-11-28 14:14:15] 1315 Done archiving!
INFO [2020-11-28 14:14:15] 1315 ---------------------------
INFO [2020-11-28 14:14:15] 1315 SUMMARY
INFO [2020-11-28 14:14:15] 1315 Processed 0 archives.
INFO [2020-11-28 14:14:15] 1315 Total API requests: 0
INFO [2020-11-28 14:14:15] 1315 done: 0 req, 773 ms, no error
INFO [2020-11-28 14:14:15] 1315 Time elapsed: 0.774s
INFO [2020-11-28 14:14:15] 1315 ---------------------------
INFO [2020-11-28 14:14:15] 1315 SCHEDULED TASKS
INFO [2020-11-28 14:14:16] 1315 Starting Scheduled tasks...
INFO [2020-11-28 14:14:16] 1315 done
INFO [2020-11-28 14:14:16] 1315 ---------------------------
@tsteur tsteur added the Regression Indicates a feature used to work in a certain way but it no longer does even though it should. label Nov 29, 2020
@tsteur tsteur added this to the 4.0.3 milestone Nov 29, 2020
@tsteur
Copy link
Member

tsteur commented Nov 29, 2020

This sounds indeed like a regression. Thanks for letting us know @kochtopf

@diosmosis I suppose when a period is forced, and we check whether we can invalidate and rearchive more imports, then we'd maybe need to look for only specific periods in that case in isInvalidationsScheduledForSite(). That query in that method will be becoming more and more complex because I suppose it would also need to include similarly queries for segments when a specific segment is forced etc. And then later it should only actually invalidate archives for that specific period (in this case day but not week, month or year) because otherwise we would have hundreds of invalidations for the same week/month/year that all need to be processed eventually even though once would be enough later potentially (not sure, depends on time_before_week_archive_considered_outdated setting etc).

@kochtopf do you mind letting us know what you configured for time_before_week_archive_considered_outdated and time_before_month_archive_considered_outdated and time_before_month_archive_considered_outdated ?

@kochtopf
Copy link
Author

sure...
time_before_month_archive_considered_outdated -1
time_before_range_archive_considered_outdated -1
time_before_today_archive_considered_outdated 900
time_before_week_archive_considered_outdated -1
time_before_year_archive_considered_outdated -1
... all set to defaults

@diosmosis diosmosis self-assigned this Nov 30, 2020
@robles-design robles-design mentioned this issue Dec 1, 2020
Closed
@diosmosis
Copy link
Member

@kochtopf this seems to work for me on 4.0.5, can you check and see if it works for you?

@kochtopf
Copy link
Author

looking good. Thank you for fixing.

@mattab mattab added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Dec 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diosmosis diosmosis

Labels
not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. Regression Indicates a feature used to work in a certain way but it no longer does even though it should.
Projects
None yet
Milestone
4.0.x
Development

No branches or pull requests
4 participants
@diosmosis @tsteur @mattab @kochtopf