New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use CLI archiving in more places for more reliable archiving (by not using the 'which' tool) #17576
Conversation
…some shells apparently do not define it.
core/CliMulti/Process.php
Outdated
@@ -210,7 +210,7 @@ public static function isSupported() | |||
return false; | |||
} | |||
|
|||
if (!self::commandExists('ps') || !self::returnsSuccessCode('ps') || !self::commandExists('awk')) { | |||
if (!self::returnsSuccessCode('ps --version') || !self::returnsSuccessCode('awk --version')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fyi haven't tested it but locally on my Mac I get this error:
ps --version
ps: illegal option -- -
usage: ps [-AaCcEefhjlMmrSTvwXx] [-O fmt | -o fmt] [-G gid[,gid...]]
[-g grp[,grp...]] [-u [uid,uid...]]
[-p pid[,pid...]] [-t tty[,tty...]] [-U user[,user...]]
ps [-L]
so this doesn't seem to work on mac.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess we can just use ps
/awk
then.
core/CliMulti/Process.php
Outdated
@@ -210,7 +210,7 @@ public static function isSupported() | |||
return false; | |||
} | |||
|
|||
if (!self::commandExists('ps') || !self::returnsSuccessCode('ps') || !self::commandExists('awk')) { | |||
if (!self::returnsSuccessCode('ps') || !self::returnsSuccessCode('awk')) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it works for the ps
, but does not work for awk
on my mac.
probably because the response for awk
is:
➜ ~/dev/sites/matomo git:(l3-81-no-which) ✗ awk
usage: awk [-F fs] [-v var=value] [-f progfile | 'prog'] [file ...]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@flamisz what is the return code for awk
on mac?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@diosmosis I'm on Debian (so GNU awk) and for me awk
also doesn't return 0 (I guess because it doesn't run without an arg)
awk ; echo $?
Usage: awk [POSIX or GNU style options] -f progfile [--] file ...
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options: GNU long options: (standard)
-f progfile --file=progfile
-F fs --field-separator=fs
-v var=val --assign=var=val
Short options: GNU long options: (extensions)
-b --characters-as-bytes
-c --traditional
-C --copyright
-d[file] --dump-variables[=file]
-D[file] --debug[=file]
-e 'program-text' --source='program-text'
-E file --exec=file
-g --gen-pot
-h --help
-i includefile --include=includefile
-l library --load=library
-L[fatal|invalid|no-ext] --lint[=fatal|invalid|no-ext]
-M --bignum
-N --use-lc-numeric
-n --non-decimal-data
-o[file] --pretty-print[=file]
-O --optimize
-p[file] --profile[=file]
-P --posix
-r --re-interval
-s --no-optimize
-S --sandbox
-t --lint-old
-V --version
To report bugs, see node `Bugs' in `gawk.info'
which is section `Reporting Problems and Bugs' in the
printed version. This same information may be found at
https://www.gnu.org/software/gawk/manual/html_node/Bugs.html.
PLEASE do NOT try to report bugs by posting in comp.lang.awk,
or by using a web forum such as Stack Overflow.
gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.
Examples:
awk '{ sum += $1 }; END { print sum }' file
awk -F: '{ print $1 }' /etc/passwd
1
So there also doesn't seem to be a POSIX command we could use.
So either we try something like awk 'BEGIN {print "ping"}'
or go back to command -v
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what about using something like awk -Wversion 2>/dev/null || awk --version
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@sgiehl's suggestion or a variant of it sounds like the best idea to me. I'll run some tests w/ my macbook
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Findus23 👍 I was specifically thinking of having multiple commands w/ ||
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Mac it awk
returns status code 1
. @diosmosis is there an issue for this? It may be acceptable that it won't work for some people where they don't have which
and we may need to change nothing since it at least mostly works for most users and may be better to not regress anything?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tsteur the issue is L3-81
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 commented there
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The command awk --version
or awk -V
works for GNU Awk, but mawk doesn't support --version or -V
@sgiehl's suggestion to use awk -Wversion 2>/dev/null || awk --version
works for both gawk and mawk for me.
core/CliMulti/Process.php
Outdated
@@ -210,7 +210,9 @@ public static function isSupported() | |||
return false; | |||
} | |||
|
|||
if (!self::commandExists('ps') || !self::returnsSuccessCode('ps') || !self::commandExists('awk')) { | |||
$awkTestCommand = 'awk -Wversion 2>/dev/null || awk --version || awk \'BEGIN {print "ping"}\' || awk'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The returnsSuccessCode()
actually appends a > /dev/null 2>&1; echo $?
to the provided command. shouldn't the commands we provide here be wrapped in brackets so the appended > /dev/null 2>&1
does not only apply to the last command?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw @diosmosis should be fine detecting AWK like this. Be good to check out comment from @sgiehl
Had just another thought before why not put the command we use awk '! /defunct/ {print $1}
in a constant and check if we get expected output?
like run echo " 537 s000 Ss 0:00.05 login -pfl theuser /bin/bash -c exec -la bash /bin/bash" | awk '! /defunct/ {print $1}'
we expect to get back 537
? This way we would know if the command is there and behaves the way we expect maybe? Not 100% sure if it's a good idea but this way might be best to detect if archiving will work. Of course no longer such a good idea if we were to use multiple / different awk
commands.
Same could be maybe done for ps
instead of checking if command exists we use ps x 2
and we check whether we get a 0 exit code since this is also what the class actually executes. I remember in past there were sometimes issues with the arguments where PS worked but not specific arguments and this way maybe would detect if we get what we expect or so.
Or maybe even easier could we simply execute getListOfRunningProcesses
and/or getRunningProcesses
and check whether we get a 0 response code (can't check for !empty maybe if no processes are detected). Just a thought to maybe simplify this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure if this was ready to be tested yet, but using the most recent version of Process.php in this PR resulted in the Managing Processes via CLI to no longer be supported on two different testing instances where it was previously working. (Ubuntu & Debian) |
core/CliMulti/Process.php
Outdated
|
||
private static function awkExistsAndRunsCorrectly() | ||
{ | ||
$testResult = `echo " 537 s000 Ss 0:00.05 login -pfl theuser /bin/bash -c exec -la bash /bin/bash" | awk '! /defunct/ {print $1}'`; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw for consistency can we use shell_exec
here as well?
fyi works nicely here (on a Mac). |
@Starker3 maybe execute below 2 commands and let us know what it outputs and the exit code:
|
This outputs a list of processes with
This outputs |
@Starker3 that's what they're supposed to return, but it's still saying it's not supported in the diagnostic? |
@diosmosis I used this version of the Process.php file when testing: https://github.com/matomo-org/matomo/blob/92256f4dcc13091c624cbe4b0f30b7b7f25dc212/core/CliMulti/Process.php |
@Starker3 would you be able to try w/ the latest? That ones missing a quote and the end of the command (which should have resulted in a fatal error). |
@diosmosis I just tested using this one: https://github.com/matomo-org/matomo/blob/a7d5f34a9f91be84ed958261c32a92d983317a81/core/CliMulti/Process.php |
@Starker3 ok, I can reproduce, looking into it. |
@diosmosis The latest changes work for me. Tested on Debian GNU/Linux 10 |
@tsteur updated again. multiple reasons displayed + constants used. |
👍 Looks good @diosmosis worked for me |
Description:
Some shells do not define
which
. For those systems, cli archiving will be unavailable and the system check will reflect this, even though everything needed to use cli archiving might be there. Fixed in this PR by just calling ps and awk and checking for a success code (which we already did for ps anyway).Review