New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ensure that the archive algorithm cannot be triggered multiple times for a same site/period/segment #1938
Comments
In #1698, hansfn suggests changing archive.sh to use /bin/sh instead of /bin/bash. I'm not opposed since /bin/sh is often symlinked to a variant, e.g., bash or dash. |
"ps aux" should work on all Linux servers. (Maybe drop the "u" since it just adds unnecessary output?) However, be aware that some times the grep command itself will be included in the ps listing - ref http://stackoverflow.com/questions/4473610 For me the "grep -v grep" addition normally works. PS! You could of course use locking files - create at start of the script, remove at the end and if killed (using trap). |
"ps aux" does work on GNU ls but certainly not on all systems. Solaris does not use GNU ls but a ls which conforms to POSIX (see http://en.wikipedia.org/wiki/Ls). We should use "ps -A" which works on all systems I got at hand (Linux, FreeBSD/NetBSD, Solaris 10). File locking as you described it can run into race conditions due to OS task scheduling. |
Note to self: Don't write comments in the middle of the night. I obviously meant GNU ps. http://en.wikipedia.org/wiki/Ps_%28Unix%29 |
This is covered in: http://stackoverflow.com/questions/185451/quick-and-dirty-way-to-ensure-only-one-instance-of-a-shell-script-is-running-at-a We just had the problem on the demo, process running 6 times in parrallel causing massive IO and crash, we should fix this..! I don't like the solution with a lock file. Instead we should use the |
matt asked me to check this issue in a comment to #2440 using ps|grep is really, really ugly... (my opinion as "shell expert") for example if some other script also named archive.sh exists on the system, and is being run, maybe even by another user... i'd strongly prefer lockfiles, and generate a warning for the user if the lockfile is locked (and of a certain age maybe)... but there's one perfectly clean and elegant solution that beats all that cruft:
such a lock is managed on the server, and automatically released when the connection is closed (script dies). (mysql has get_lock(name,timeout) ) the only issue here is that the archive job opens a new database connection (opened by a separate api/php invocation) for each processing step... i would suggest to rewrite the bottom part of the script in php and move it into the piwik api as a function, that would also help reduce the language diversity, and the need for shell programmers, which seem to be lacking in piwik project staff... ;) |
Replying to halfdan:
wrong! not if done correctly. what is needed is some kind of atomic operation that can only succeed once... the by far easiest alternative in a shellscript is mkdir though, which i prefer to use for locking. if ! mkdir /tmp/mylock ; then echo failed to get lock exit 1 fi trap "rmdir /tmp/mylock" EXIT a bunch of stuff worth reading: |
i updated my rework of the script for #2440 to use a lockfile. |
I agree that the DB lock would be good to have, but like you say it is nice to do it in the script also ;-) Thanks for implementing lock file! Maybe when a lock file was detected, the error message could say "If no other process is running, you can manually delete the lock file at: $URL" to help users know what to do next if for some reasons lock wasn't deleted (server crashed maybe?) Otherwise it's great, I think the next step for improving the script would be, multithread the requests to make use of the cores of a system ;) See also #2440 |
Replying to matt:
that would be possible to add, with a little restructuring... |
Running in parallel will increase performance, because the data queried will be different, and all the time spent in PHP processing can be multithreaded very efficiently. It might not work well for very large websites, but will make a huge difference in the use case of a piwik with thousands of small websites to process. |
For the record, using a multithreaded archive.sh (from #2563), performance increases almost linearly. I have a 8-core machine and more than 20,000 sites in my Piwik. Having a lock on archive.sh doesn't seem to be an issue to me: multithreaded is handled by xargs running inside archive.sh. |
I don't know if this will help or hinder, but in [5065], we are now using get_lock/release_lock around the archive processing. |
vipsoft, Re: [5065] The release lock name I think should contain the segment as well: $this->segment since you can archive several segments for a same site/period. Adding here as a note to check before release. |
re: lock name. Feel free to add segment to the name. |
(In [5102]) Refs #2327
|
@tsteur thanks for the report. Could you create a new issue with your
screenshot?
|
Is it a different issue than this one? |
It's the same issue, but because it has regressed we usually create a new issue |
If the script is already running (
ps aux | grep archive.sh | wc -l
or similar) if it is running, script echos "archive.sh already running, exiting..."is there any reason the simple ps aux would not work on all linux servers?
Ideally the windows powershell version should do this as well, but optional for now
The text was updated successfully, but these errors were encountered: