Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bad archiver behavior #6977

Closed
DaSchTour opened this issue Jan 10, 2015 · 3 comments
Closed

Bad archiver behavior #6977

DaSchTour opened this issue Jan 10, 2015 · 3 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@DaSchTour
Copy link
Contributor

Well I discovered some bad behavior of the archive script. The first problem is, that the logfile is overwritten. This means, that there is no possibility to check if there was any broken archiving.
The next problem is, that if some parts are not archived because of inconsistencies, this isn't seen as an error and not reported as such. The next thing is, that the script sometimes seems to correctly run through without finishing the logfile. The data is correctly stored but the logfile is doesn't end with finish. Which maybe leeds to the inconstancies which are not reported. So I have to watch the logfiles manually if there is anything wrong. So I also could start archiving manually which would result in the same amount of work. I would like to post some logfile, but as they are deleted when the archiver is stared again I have non of them.
@DaSchTour
Copy link
Contributor Author

Additionally there seems to be a problem with the following:
When this event occurs: "Will ignore websites and help finish a previous started queue instead. IDs: 6" the next archive run will starts with: "Will process 1 websites with new visits since 29s , IDs: 1" which is somehow stupid, because this page might not have been processed for weeks because it was hanging on queue for ID:6 which than might lead to the problem, that some week or month reports are not available. Also a problem which is randomly discovered and is only fixable by deleting the logs and creating completely new, which takes a lot of time and than also leads to problems and interference with the cron tasks.

@gaumondp
Copy link

I don't know if I misunderstand the comment about log being overwritten but here is how I keep it grow using the cron job :

*/15 * * * * php /piwik/console core:archive --url=http://1.2.3.4 >> /logs/piwik-cron.log

Using ">>" make the log grow but you keep the full history.

@mattab
Copy link
Member

mattab commented Feb 19, 2015

can you create an issue for each problem, i'm a bit lost here + see @gaumondp comment for why the log was overwritten (we do this in the doc so that users don't end up with a log file of several Gb)

@mattab mattab closed this as completed Feb 19, 2015
@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label Feb 19, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mattab @DaSchTour @gaumondp