Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

core:archive may show "no new tracking data for website id since the last successful archiving" even when last successful archiving didn't archive this site #15393

Closed
mattab opened this issue Jan 14, 2020 · 2 comments
Closed

core:archive may show "no new tracking data for website id since the last successful archiving" even when last successful archiving didn't archive this site #15393

mattab opened this issue Jan 14, 2020 · 2 comments
Labels
answered For when a question was asked and we referred to forum or answered it. Bug For errors / faults / flaws / inconsistencies etc.
Milestone
Priority Backlog ...

Comments

@mattab
Copy link
Member

mattab commented Jan 14, 2020

Below is an example of two core:archive runs, one after another. The first one doesn't process the idsite = 1 because it Will ignore websites and help finish a previous started queue instead. IDs: 2.

Then the second archiving says Archiving was last executed without error 46s ago and no new tracking data for website id 1 since 2020-01-14 23:09:09 UTC (since the last successful archiving) although the previous successful archiving did not archive idsite=1. Instead we expected that it would look for "new tracking data" since the time where idsite=1 was actually last archived.

Proposal:

  • consider the time of "previous successful archiving for this specific idsite" when looking if there were new tracking data since that time.
INFO [2020-01-14 23:08:12] 16707  ---------------------------
INFO [2020-01-14 23:08:12] 16707  INIT
INFO [2020-01-14 23:08:12] 16707  Running Matomo 3.13.1-rc1 as Super User
INFO [2020-01-14 23:08:12] 16707  ---------------------------
INFO [2020-01-14 23:08:12] 16707  NOTES
INFO [2020-01-14 23:08:12] 16707  - Async process archiving supported, using CliMulti.
INFO [2020-01-14 23:08:12] 16707  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-01-14 23:08:12] 16707  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-01-14 23:08:12] 16707  - Archiving was last executed without error 57 days 22 hours ago
INFO [2020-01-14 23:08:12] 16707  - Will process 3 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 1, 2, 3
INFO [2020-01-14 23:08:12] 16707  - Will process 1 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 3
INFO [2020-01-14 23:08:12] 16707  Will ignore websites and help finish a previous started queue instead. IDs: 2
INFO [2020-01-14 23:08:12] 16707  ---------------------------
INFO [2020-01-14 23:08:12] 16707  START
INFO [2020-01-14 23:08:12] 16707  Starting Matomo reports archiving...
INFO [2020-01-14 23:08:12] 16707  Day has finished for website id 2 since last run
INFO [2020-01-14 23:08:13] 16707  Will pre-process for website id = 2, period = day, date = last52
INFO [2020-01-14 23:08:13] 16707  - pre-processing all visits
INFO [2020-01-14 23:08:24] 16707  Archived website id = 2, period = day, 0 segments, 0 visits in last 52 days, 0 visits today, Time elapsed: 11.322s
INFO [2020-01-14 23:08:24] 16707  Will pre-process for website id = 2, period = week, date = last59
INFO [2020-01-14 23:08:24] 16707  - pre-processing all visits
INFO [2020-01-14 23:08:51] 16707  Archived website id = 2, period = week, 0 segments, 0 visits in last 59 weeks, 0 visits this week, Time elapsed: 27.592s
INFO [2020-01-14 23:08:54] 16707  Will pre-process for website id = 2, period = month, date = last52
INFO [2020-01-14 23:08:54] 16707  - pre-processing all visits
INFO [2020-01-14 23:08:59] 16707  Archived website id = 2, period = month, 0 segments, 0 visits in last 52 months, 0 visits this month, Time elapsed: 7.830s
INFO [2020-01-14 23:09:00] 16707  Will pre-process for website id = 2, period = year, date = last7
INFO [2020-01-14 23:09:00] 16707  - pre-processing all visits
INFO [2020-01-14 23:09:02] 16707  Archived website id = 2, period = year, 0 segments, 0 visits in last 7 years, 0 visits this year, Time elapsed: 2.731s
INFO [2020-01-14 23:09:02] 16707  Archived website id = 2, 4 API requests, Time elapsed: 49.511s [1/1 done]
INFO [2020-01-14 23:09:02] 16707  Done archiving!
INFO [2020-01-14 23:09:02] 16707  ---------------------------
INFO [2020-01-14 23:09:02] 16707  SUMMARY
INFO [2020-01-14 23:09:02] 16707  Total visits for today across archived websites: 0
INFO [2020-01-14 23:09:02] 16707  Archived today's reports for 1 websites
INFO [2020-01-14 23:09:02] 16707  Archived week/month/year for 1 websites
INFO [2020-01-14 23:09:02] 16707  Skipped 2 websites
INFO [2020-01-14 23:09:02] 16707  - 0 skipped because no new visit since the last script execution
INFO [2020-01-14 23:09:02] 16707  - 0 skipped because existing daily reports are less than 900 seconds old
INFO [2020-01-14 23:09:02] 16707  - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2020-01-14 23:09:02] 16707  Total API requests: 4
INFO [2020-01-14 23:09:02] 16707  done: 1/1 100%, 0 vtoday, 1 wtoday, 1 wperiods, 4 req, 49731 ms, no error
INFO [2020-01-14 23:09:02] 16707  Time elapsed: 49.731s
INFO [2020-01-14 23:09:02] 16707  ---------------------------
INFO [2020-01-14 23:09:02] 16707  SCHEDULED TASKS
INFO [2020-01-14 23:09:02] 16707  Starting Scheduled tasks... 

[....]


$ ./console core:archive
INFO [2020-01-14 23:09:55] 18931  ---------------------------
INFO [2020-01-14 23:09:55] 18931  INIT
INFO [2020-01-14 23:09:55] 18931  Running Matomo 3.13.1-b1 as Super User
INFO [2020-01-14 23:09:55] 18931  ---------------------------
INFO [2020-01-14 23:09:55] 18931  NOTES
INFO [2020-01-14 23:09:55] 18931  - Async process archiving supported, using CliMulti.
INFO [2020-01-14 23:09:55] 18931  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-01-14 23:09:55] 18931  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-01-14 23:09:55] 18931  - Archiving was last executed without error 46s ago
INFO [2020-01-14 23:09:55] 18931  - Will process 1 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 3
INFO [2020-01-14 23:09:55] 18931  ---------------------------
INFO [2020-01-14 23:09:55] 18931  START
INFO [2020-01-14 23:09:55] 18931  Starting Matomo reports archiving...
INFO [2020-01-14 23:09:55] 18931  - The following websites do not use the tracker: 3
INFO [2020-01-14 23:09:55] 18931  - no new tracking data for website id 1 since 2020-01-14 23:09:09 UTC (since the last successful archiving)
INFO [2020-01-14 23:09:55] 18931  Skipped website id 1 as archiving is not needed
INFO [2020-01-14 23:09:55] 18931  - no new tracking data for website id 2 since 2020-01-14 23:09:09 UTC (since the last successful archiving)
INFO [2020-01-14 23:09:55] 18931  Skipped website id 2 as archiving is not needed
INFO [2020-01-14 23:09:55] 18931  Old report was invalidated for website id 3
INFO [2020-01-14 23:09:55] 18931  Will pre-process for website id = 3, period = day, date = last52
INFO [2020-01-14 23:09:55] 18931  - pre-processing all visits
INFO [2020-01-14 23:10:09] 18931  Archived website id = 3, period = day, 0 segments, 4 visits in last 52 days, 1 visits today, Time elapsed: 13.875s
INFO [2020-01-14 23:10:09] 18931  Will pre-process for website id = 3, period = week, date = last206
INFO [2020-01-14 23:10:09] 18931  - pre-processing all visits
INFO [2020-01-14 23:11:22] 18931  Archived website id = 3, period = week, 0 segments, 16 visits in last 206 weeks, 1 visits this week, Time elapsed: 72.509s
INFO [2020-01-14 23:11:22] 18931  Will pre-process for website id = 3, period = month, date = last52
INFO [2020-01-14 23:11:22] 18931  - pre-processing all visits
@mattab mattab added the Bug For errors / faults / flaws / inconsistencies etc. label Jan 14, 2020
@mattab mattab added this to the 4.2.0 milestone Jan 14, 2020
@tsteur
Copy link
Member

tsteur commented Jan 14, 2020

Duplicate of #15127 ?

Even using a time of a specific idsite might not work reliably since some segments might be processed but not others.

@mattab
Copy link
Member Author

mattab commented Dec 11, 2023

As it has been a few months since the last activity and we believe this is likely not an issue anymore, we will now close this. If that's not the case, please do feel free to either reopen this issue or open a new one. We will gladly take a look again!

@mattab mattab closed this as not planned Won't fix, can't repro, duplicate, stale Dec 11, 2023
@sgiehl sgiehl added the answered For when a question was asked and we referred to forum or answered it. label Dec 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
answered For when a question was asked and we referred to forum or answered it. Bug For errors / faults / flaws / inconsistencies etc.
Projects
None yet
Milestone
Priority Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@tsteur @mattab @sgiehl