Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make import_logs.py IPv6 compatible #4062

Closed
anonymous-matomo-user opened this issue Jul 30, 2013 · 6 comments
Closed

Make import_logs.py IPv6 compatible #4062

anonymous-matomo-user opened this issue Jul 30, 2013 · 6 comments
Labels
duplicate For issues that already existed in our issue tracker and were reported previously. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Milestone
Long term

Comments

@anonymous-matomo-user
Copy link

Hi,

we are currently investigating if piwik logfile analytics can replace awstats log file analytics.
Doing so we came across the problem that piwik seems not to be able to parse log files with IPv6 addresses. Removing every IPv6 log line from file and the import works fine.

The following import command is used

python /var/www/piwik.ibumedia.de/misc/log-analytics/import_logs.py --url=http://piwik.ibumedia.de --idsite=77 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --recorder-max-payload-size=400 --dry-run 20130728.log

The log format is an iis 7 w3c format with all fields selected.
Lines like this will fail on import

2013-07-28 02:34:09 W3SVC01 HOST123 1234:1234:12:123::1 GET / - 80 - 2a01:4f8:0:a101::6:1 HTTP/1.1 Hetzner+System+Monitoring - - www.example.com 200 0 0 3446 96 0

best regards
mr.moe
Keywords: logfile ipv6 import
@mattab
Copy link
Member

mattab commented Aug 9, 2013

It would be awesome if you could submit a Pull request or patch for fixing log import script to work with IPv6 addresses! http://piwik.org/participate/contributing-with-git/

@mattab
Copy link
Member

mattab commented Feb 1, 2014

Consolidating milestones FTW

@mattab
Copy link
Member

mattab commented Mar 16, 2014

It is getting more important to get IPv6 support in log import.

Thoughts:

  • Maybe the IPs Regexes for the various formats can be updated to also accept ipv6?
  • add unit tests (ie. one ipv6 line in each log-analytics/tests/*.xml)
  • add integration tests (ie. one ipv6 in each format in /tests/resources/access-logs/*.log)

@anonymous-matomo-user anonymous-matomo-user added this to the 2.x - The Great Piwik 2.x Backlog milestone Jul 8, 2014
@mattab mattab removed the Major label Aug 3, 2014
@multiformeingegno
Copy link

Is this in some roadmap? Any schedule?

@mattab
Copy link
Member

mattab commented Sep 9, 2014

@multiformeingegno not yet scheduled, please help with a pull request, if you can!

@mattab mattab closed this as completed Sep 9, 2014
@mattab mattab reopened this Sep 9, 2014
Closed
@mattab
Copy link
Member

mattab commented Mar 12, 2015

Issue was moved to the new repository for Piwik Log Analytics: https://github.com/piwik/piwik-log-analytics/issues

refs #7163

@mattab mattab closed this as completed Mar 12, 2015
@mattab mattab added the duplicate For issues that already existed in our issue tracker and were reported previously. label Mar 12, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
duplicate For issues that already existed in our issue tracker and were reported previously. Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc.
Projects
None yet
Milestone
Long term
Development

No branches or pull requests
3 participants
@mattab @multiformeingegno @anonymous-matomo-user