Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can I extract log according to --include-path configuration? #11948

Open
triffic-tang opened this issue Aug 9, 2017 · 4 comments
Open

Can I extract log according to --include-path configuration? #11948

triffic-tang opened this issue Aug 9, 2017 · 4 comments

Comments

@triffic-tang
Copy link

Hi, there
The access log of apache shown below:

109.6.29.10 - - [28/Jul/2017:14:57:30 +0800] "GET /s3Core/~main/share/frame.php?lefttree=false&target=importMessage HTTP/1.0" 200 19706 "http://xxx.xxx.com/s3demosy/~main/share/frame.php?lefttree=false&target=syncMessage" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36"
109.6.29.10 - - [28/Jul/2017:14:57:30 +0800] "POST /s3Core/~main/ajax.php HTTP/1.0" 200 276 "http://xxx.xxx.com/s3demosy/~main/share/frame.php?lefttree=false&target=importMessage" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36"
109.6.29.10 - - [28/Jul/2017:14:57:30 +0800] "POST /s3Core/~main/ajax.php HTTP/1.0" 200 849 "http://xxx.xxx.com/s3demosy/~main/share/frame.php?lefttree=false&target=importMessage" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.132 Safari/537.36"
109.6.29.10 - - [28/Jul/2017:15:02:27 +0800] "POST /s3Core/~main/ajax.php HTTP/1.0" 200 221 "http://xxx.xxx.com/s3demotest/mobile/login.php" "Mozilla/5.0 (iPhone; CPU iPhone OS 8_0 like Mac OS X) AppleWebKit/600.1.3 (KHTML, like Gecko) Version/8.0 Mobile/12A4345d Safari/600.1.4"

Pay attention to that there are two websites http://xxx.xxx.com/s3demotest and http://xxx.xxx.com/s3demosy's access log.

Then command below taken executed. I want let piwik Log Analytics Tool process specified log line(s) according to --include-path configuration, but it seems does't work.

COMMAND

python import_logs.py --url=http://piwik.gyj.com/piwik/ --login=xxx --password=xxx --enable-http-errors --enable-http-redirects --enable-static --enable-bots --include-path="http://xxx.xxx.com/s3demotest/" --idsite=4 /home/webapp/piwik/logs/access_log_s3.log

image

I think I may wrong on the usage of the --include-path configuration

--include-path=INCLUDE_PATHS
Paths to include, Can be specified multiple times. If not specified, all paths are included.

Can someone tell me how to figure out why?

@sgiehl
Copy link
Member

sgiehl commented Aug 9, 2017

Did you try to use the path only, without domain. e.g. --include-path=s3demotest (maybe with or without trailing/leading slash)

@triffic-tang
Copy link
Author

Thank you for response @sgiehl .
I tried:

python import_logs.py --url=http://piwik.gyj.com/piwik/ --login=xxx --password=xxx --enable-http-errors --enable-http-redirects --enable-static --enable-bots --include-path="/s3demotest*" --idsite=4 /home/webapp/piwik/logs/access_log_s3.log

But it still gives nothing.

@Findus23
Copy link
Member

Findus23 commented Aug 9, 2017

It seems like http://piwik.gyj.com/piwik/ still doesn't point to an piwik instance. (#11943)

Are you sure you have set it up correctly?

@triffic-tang
Copy link
Author

triffic-tang commented Aug 9, 2017

Yes, Have set it up correctly.

COMMAND

python import_logs.py --url=http://piwik.gyj.com/piwik/ --login=xxx --password=xxx --enable-http-errors --enable-http-redirects --enable-static --enable-bots --exclude-path="/s3demotest*" --idsite=4 /home/webapp/piwik/logs/test.log

Test.log contains 7 items, just for test.

RESULTS WITH --EXCLUDE-PATH

image

@Findus23 , but thanks for replying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants