of the archive cronjob.
This can be helpfull when you approach the memory limit.
The build is failing for reason that are unrelated to this PR.
All the tests that this PR broke are fixed.
Thanks for the PR. Started reviewing it but then realised that this script is not actually the script which runs the archiving tasks. They are either
In both cases the memory of the archiving script will not raise much as the actual memory hungry processes are in separate threads.
Therefore I would say this PR is not needed. Could you clarify whether it was helpful for you at some point, and why?
@mikeSimonson Please see my last comment for questions. For now closing the PR but happy to re-open if it's actually useful :+1:
@mikeSimonson would you mind sending here an example of the archive log output which shows the memory increase/decrease or another interesting use case? Thanks
INFO [2017-08-27 22:40:19] Archived website id = 3, period = year, 0 segments, 1495404 visits in last 2 years, 1495404 visits this year, Time elapsed: 45.997s INFO [2017-08-27 22:40:19] Archived website id = 3, 4 API requests, Time elapsed: 35.564s [3/9 done] INFO [2017-08-27 22:40:19] Mermory usage: 123.55MB INFO [2017-08-27 22:40:19] - no new tracking data for website id 4 since 2017-08-27 22:35:11 UTC (since the last successful archiving) INFO [2017-08-27 22:40:19] Skipped website id 4 as archiving is not needed