Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importing apache2-logs with /piwik/misc/log-analytics/import_logs.py #15242

Closed
Kornexl opened this issue Dec 6, 2019 · 10 comments
Closed

importing apache2-logs with /piwik/misc/log-analytics/import_logs.py #15242

Kornexl opened this issue Dec 6, 2019 · 10 comments
Labels
answered For when a question was asked and we referred to forum or answered it.

Comments

@Kornexl
Copy link

Kornexl commented Dec 6, 2019

I’m importing Apache logs since some years and it always worked OK.
Since Update to Matome 3.13.0 on 28.11. the number of unique visitors is way to high

The number of loglines the days before are about at the same magnitude, but the number of visitors was suddenly a huge factor higher (500 vs 20.000)
Number of lines in these Logs
44486 ezproxy.log-20191126 (468 visitors)
50920 ezproxy.log-20191127 (498 visitors)
44860 ezproxy.log-20191128 (27.11.2019 and 491 visitors)
41085 ezproxy.log-20191129 (28.11.2019 and 18000 visitors)
42425 ezproxy.log-20191130
30908 ezproxy.log-20191201
29724 ezproxy.log-20191202
42316 ezproxy.log-20191203 (28.179 visitors)

@Kornexl
Copy link
Author

Kornexl commented Dec 6, 2019

docweb-besucher

@tsteur
Copy link
Member

tsteur commented Dec 6, 2019

How exactly do you execute the log import command with which parameters? Do the requests have a userId?

@mattab
Copy link
Member

mattab commented Dec 11, 2019

@Kornexl Could you please try this patch https://github.com/matomo-org/matomo/pull/15258/files
and let us know if this maybe helps with counting visits correctly (for new visits after applying the patch)?

@psec
Copy link

psec commented Dec 11, 2019

@mattab experiencing a similar bug and the patch posted above did not fix it.

@Kornexl
Copy link
Author

Kornexl commented Dec 17, 2019

The logs do have userids and these are transmitted in the uid-Parameter by piwik/misc/log-analytics/import_logs.py
Prior to 3.13 different userids are counted as the visits
Now the visits don't use the userids but count otherwise.
Nearly every used logline is counted as a visit even when the same userid (same IP, same user agent) clicked on another link one minute later.
The different userids are shown as users in the widgets
That's a breaking change in the semantics of visits

@Kornexl
Copy link
Author

Kornexl commented Dec 18, 2019

by importing logs there is no cookie set

@mattab
Copy link
Member

mattab commented Dec 24, 2019

@Kornexl can you please share a short extract from the log with multiple lines for the same user ID resulting in multiple visits? if you can't share here publicly, feel free to send us by email at hello at matomo.org - please also include the URL to this issue #15242 - we'll investigate further

@Kornexl
Copy link
Author

Kornexl commented Jan 17, 2020

should this problem be solved with the new release 3.13.1?
I have installed this release today and can check tomorrow after the import of the daily log

@Kornexl
Copy link
Author

Kornexl commented Jan 19, 2020

This problem is resolved with 3.13.1

@Kornexl Kornexl closed this as completed Jan 19, 2020
@mattab
Copy link
Member

mattab commented Jan 21, 2020

Thanks for confirming @Kornexl 👍

@mattab mattab added the answered For when a question was asked and we referred to forum or answered it. label Feb 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
answered For when a question was asked and we referred to forum or answered it.
Projects
None yet
Development

No branches or pull requests

4 participants