Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

import_logs.py - STDIN doesn't works #16474

Closed
anthosz opened this issue Sep 25, 2020 · 5 comments
Closed

import_logs.py - STDIN doesn't works #16474

anthosz opened this issue Sep 25, 2020 · 5 comments
Labels
duplicate For issues that already existed in our issue tracker and were reported previously.

Comments

@anthosz
Copy link

anthosz commented Sep 25, 2020 •
edited

Hi,

I use matomo 3.14.1 and I cannot use STDIN.

If I try (without stdin, it works):

python import_logs.py --idsite=1 --url='XXXXXX' --recorders=3 --
enable-http-errors --enable-http-redirects --enable-static --enable-bots --show-progress --show-progress-delay=10 --log-format-regex="(?P<ip>\S+) \[(?P<date>.*?) (?P<timezone>.*?)\] \"\S+ (
?P<path>.*?)\" (?P<status>.*)" access
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log access...
^C
Logs import summary
-------------------
    968 requests imported successfully

If I try with this method (stdin method 1 / NOK):

# python import_logs.py --idsite=1 --url='XXXXXX' --recorders=3 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --show-progress --show-progress-delay=10 --log-format-regex="(?P<ip>\S+) \[(?P<date>.*?) (?P<timezone>.*?)\] \"\S+ (?P<path>.*?)\" (?P<status>.*)" - < access
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Logs import summary
-------------------
    0 requests imported successfully

If I try with this method (STDIN method 2 / NOK):

# cat access | python import_logs.py --idsite=1 --url='XXXXXX' --recorders=3 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --show-progress --show-progress-delay=10 --log-format-regex="(?P<ip>\S+) \[(?P<date>.*?) (?P<timezone>.*?)\] \"\S+ (?P<path>.*?)\" (?P<status>.*)" -
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Logs import summary
-------------------
    0 requests imported successfully

Seems that the file is empty according to output but it's full of lines (according of the test without stdin that works)...

Do you have an idea?

Best regards,
@tsteur
Copy link
Member

tsteur commented Sep 26, 2020

Hi @anthosz are you using the latest version? I think this was fixed in matomo-org/matomo-log-analytics#275

@anthosz
Copy link
Author

anthosz commented Sep 26, 2020

Hi @tsteur,

I confirm that yes, I use version 3.14.1.

Bestrehards,

@tsteur
Copy link
Member

tsteur commented Sep 27, 2020

Thanks @anthosz I just checked and the fix was not included in the last release. It was fixed in matomo-org/matomo-log-analytics#275

You could apply the patch by updating your importer by overwriting the import_logs.py with this content: https://raw.githubusercontent.com/matomo-org/matomo-log-analytics/3.x-dev/import_logs.py or by applying these changes: https://github.com/matomo-org/matomo-log-analytics/pull/275/files

@tsteur tsteur closed this as completed Sep 27, 2020
@tsteur tsteur added the duplicate For issues that already existed in our issue tracker and were reported previously. label Sep 27, 2020
@anthosz
Copy link
Author

anthosz commented Sep 27, 2020

Thank you @tsteur for your reactivity!

I will do it tomorrow!

@anthosz
Copy link
Author

anthosz commented Sep 28, 2020

I confirm, once patched, it works!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate For issues that already existed in our issue tracker and were reported previously.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tsteur @anthosz