New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
./import_logs.py fatal error processing Jetty request logs (log files with multiple spaces) #7228
Comments
If I use the latest log importer script, I can successfully import these visits (log importer output is below). Can you try importing w/ the latest script: https://raw.githubusercontent.com/piwik/piwik/master/misc/log-analytics/import_logs.py Log importer output:
|
Still fails. bash-4.1$ ./import_logs.py --url=http://localhost/piwik --idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --token-auth=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 2015_02_16.request.log I would attach the entire log file that fails but "Unfortunately, we don't support that file type." when I try to. |
Sorry, closed by accident. |
Does it work w/ the subset of logs you posted in the ticket? Ie, if you copy-paste them to a file and run the importer w/ just those logs, will it run, successfully? |
Yes, that works. The entire log comprises much more and that fails. On Wed, Feb 18, 2015 at 11:38 AM, Benaka notifications@github.com wrote:
|
@degenaro then can you paste us a log that fails, so we can reproduce the issue? |
I tried to cut and paste here (which was tedious due to cut and paste buffer size?), then got an error when trying to close and comment because the amount of data was too large? The file comprising the log is 1448 lines long. |
@degenaro post the logs on http://pastebin.com/ or another similar site? |
[log files redacted] |
The above log is 412 lines. It fails for me. |
There are only 305 lines in your comment (it is cut off at the end), and they correctly parsed by the latest log importer. Please use an external service (like dropbox or pastebin) to post the log, or email an archive to hello@piwik.org. |
Failing log sent via e-mail. |
I received the logs and can reproduce the error, will post here when I find the cause. |
Ok, the error is due to spaces in the log lines. Some fields are separated by two spaces instead of one and the log importer can't handle that (for non W3C extended log formats). Working on a fix. |
…ual log fields. Includes python tests.
…ual log fields. Includes python tests.
When will the fix appear in a release and how do I get the fix between now Thanks. Lou. On Sun, Mar 1, 2015 at 11:29 PM, Matthieu Aubry notifications@github.com
|
Use the latest file in master, ie, https://raw.githubusercontent.com/piwik/piwik/master/misc/log-analytics/import_logs.py |
See http://forum.piwik.org/read.php?2,124212
10.199.199.10 - - [16/Feb/2015:10:43:45 +0000] "GET /jobs.jsp HTTP/1.1" 200 0 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/cluster-name HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/version HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/login-link HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/logout-link HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/classic-jobs-data HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /js/ducc.local.js?=1424083427017 HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /js/ducc.local.js?=1424083427018 HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
10.199.199.10 - - [16/Feb/2015:10:43:47 +0000] "GET /ducc-servlet/authenticator-version HTTP/1.1" 200 0 "http://192.168.6.67:42133/jobs.jsp" "Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Firefox/31.0"
The text was updated successfully, but these errors were encountered: