Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrong parameters (date range) used when finishing a previously started queue #8261

Closed
patriiiiiiiiiick opened this issue Jul 1, 2015 · 8 comments
Closed

Wrong parameters (date range) used when finishing a previously started queue #8261

patriiiiiiiiiick opened this issue Jul 1, 2015 · 8 comments
Labels
Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Milestone
Backlog (Help wan...

Comments

@patriiiiiiiiiick
Copy link

Run:
./console core:archive --force-all-websites --force-date-range=2015-06-01,2015-06-25
which for some reason quits in the middle of the list of 20 sites.

Later on, run:
./console core:archive --force-all-websites --force-date-range=2015-05-01,2015-05-31
the output of which gives
Will ignore websites and help finish a previous started queue instead. IDs: 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
while the rest of the output will be like
Archived website id = 15, period = year, 4 segments, 1352807 visits in years included in: 2015-05-01,2015-05-31, Time elapsed: 495.273s

I hereby conclude that the previous job was finished but with the new parameters (date range). It means that my instruction to archive May was only half honoured while the one to archive June was never actually finished.
@tsteur
Copy link
Member

tsteur commented Jul 1, 2015

I'm not quite sure if I understand. I think one bug might be that if someone uses --force-all-websites we should not process a previously started queue. This might have been regressed.

Also the queue should maybe not used in general if a --force-data-range is given since it would be not really correct. Eg the queue might have been started with a different data range.

@mattab
Copy link
Member

mattab commented Jul 15, 2015

I think one bug might be that if someone uses --force-all-websites we should not process a previously started queue. This might have been regressed.

This was actually changed on purpose, see: #7682 and #7614

Also the queue should maybe not used in general if a --force-data-range is given since it would be not really correct.

Good point! I think it is the right fix, to not use the shared queue when --force-date-range and --force-all-websites are used at the same time.

Maybe there are other parameters that when set, should not use a shared queue?

@mattab mattab added the Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. label Jul 15, 2015
@mattab mattab added this to the 2.15.0 milestone Jul 15, 2015
@tsteur
Copy link
Member

tsteur commented Jul 16, 2015

This was actually changed on purpose, see: #7682 and #7614

I don't think it's actually good to do it this way. I'd expect --force-all-websites not to resume the queue. At least it should reset it or so as that behaviour can be buggy in some ways.

@mattab mattab removed this from the 2.15.0 milestone Jul 22, 2015
@tsteur
Copy link
Member

tsteur commented Aug 13, 2015

By the way there is also a related problem that whenever we run the archiver we set the archiving time at the end: https://github.com/piwik/piwik/blob/2.14.3/core/CronArchive.php#L399

This means when executing the archiver the next time we will just look for that lastSuccesRunTime but we may have actually only processed one site at lastSuccessRunTimestamp, all other sites have a completely different lastSuccessRunTimestamp. This can also lead to some side effects when using multiple archivers at the same time. When using force-siteids etc we should maybe not set the lastSuccessRunTime, only when actually all websites were processed.

@tsteur
Copy link
Member

tsteur commented Aug 13, 2015

Maybe it might be worth having a look at this in 2.15 as it will be the stable version for a year @mattab

@mattab
Copy link
Member

mattab commented Aug 13, 2015

@tsteur thanks for suggestion, moving to 2.15.0 👍

Let me know how you see a fix for this, or we can discuss in the PR.

@tsteur
Copy link
Member

tsteur commented Aug 13, 2015

I'm not working on this one

@mattab mattab added this to the 2.15.1 milestone Sep 18, 2015
@mattab mattab modified the milestones: Short term, 2.15.1 Nov 18, 2015
@mattab mattab mentioned this issue Apr 11, 2016
Closed
@mattab
Copy link
Member

mattab commented Dec 14, 2023

Thanks for contributing to this issue. As it has been a few months since the last activity and we believe this is likely not an issue anymore, we will now close this. If that's not the case, please do feel free to either reopen this issue or open a new one. We will gladly take a look again!

@mattab mattab closed this as not planned Won't fix, can't repro, duplicate, stale Dec 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change.
Projects
None yet
Milestone
Backlog (Help wanted)
Development

No branches or pull requests
3 participants
@tsteur @mattab @patriiiiiiiiiick