Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem in import or archive log #4295

Closed
anonymous-matomo-user opened this issue Nov 12, 2013 · 15 comments
Closed

Problem in import or archive log #4295

anonymous-matomo-user opened this issue Nov 12, 2013 · 15 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Milestone

Comments

@anonymous-matomo-user
Copy link

Hello,

Since the upgrade in version 2.0b3, i have a problem in the import log process (import_logs.py) or in the generated report process (archive.sh). My reports seems to not include all the HTTP requests.

I import my HTTP logs with this options :
--enable-http-errors
--enable-http-redirects
--enable-static
--enable-bots

When i look my reports in Piwik, only requests on static elements are computed.

In the "Visitor log" menu, i can see all the visitors but with 0 action. I see only the requests on GIF, JPG...

There's no error message during the process of my logs files.

Thx
Stphane

@anonymous-matomo-user
Copy link
Author

This is the logs import summary

723073 lines parsed, 723073 lines recorded, 2363 records/sec (avg), 10129 records/sec (current)
Purging Piwik archives for dates: 2013-11-10 2013-11-11
To re-process these reports with your new update data, execute the piwik/misc/cron/archive.php script, or see: http://piwik.org/setup-auto-archiving/ for more info.

Logs import summary

723073 requests imported successfully
105808 requests were downloads
0 requests ignored:
    0 invalid log lines
    0 requests done by bots, search engines, ...
    0 HTTP errors
    0 HTTP redirects
    0 requests to static resources (css, js, ...)
    0 requests did not match any known site
    0 requests did not match any requested hostname

Website import summary

723073 requests imported to 1 sites
    1 sites already existed
    0 sites were created:

0 distinct hostnames did not match any existing site:

Performance summary

Total time: 305 seconds
Requests imported per second: 2363.16 requests per second

@mattab
Copy link
Member

mattab commented Nov 14, 2013

Can you reproduce the issue with a small log file? could you send us the small log file or attach it to this ticket?

what command line do you use exactly?

@anonymous-matomo-user
Copy link
Author

Attachment:
access.zip

@anonymous-matomo-user
Copy link
Author

Yes, i can reproduce the issue with the file attachment

Here the command line

misc/log-analytics/import_logs.py --url=http://mypiwikurl --idsite=12 --recorders=24 --enable-http-errors --enable-http-redirect --enable-static --enable-bots --recorder-max-payload-size=400 /tmp/access.log


@mattab
Copy link
Member

mattab commented Nov 15, 2013

I imported the log file and it works for me on 2.0-beta5 (see screenshot).

could you please

  • create a new website in piwik
  • import the log file with 12 lines that you uploaded here
  • post the screenshot of the visitor log in this new website for these 12 requests

@mattab
Copy link
Member

mattab commented Nov 15, 2013

Attachment:
works.png

@anonymous-matomo-user
Copy link
Author

Hi,

I upgraded my Piwik in 2.0-beta5, created a new site and imported the 12 requests. The result is the same.

Please, see the attachment files.

@anonymous-matomo-user
Copy link
Author

Attachment:
Dashboard.JPG

@anonymous-matomo-user
Copy link
Author

Attachment:
VisitorsLog.JPG

@anonymous-matomo-user
Copy link
Author

Strange thing ! In the Dashboard, the 5 visites are dated 13/11 and in the Visitors Log they are dated 14/11. The true date is 14/11

Stphane

@mattab
Copy link
Member

mattab commented Nov 15, 2013

I cannot reproduce. can you please create a new piwik installation of piwik 2.0-beta5, then import the 12 requests into site id = 1, then export the full mysql database and send to matt@piwik.org ? I'll take a look if pages are not tracked there. Thanks

@anonymous-matomo-user
Copy link
Author

I make a fresh install of Piwik and import the 12 requests.
The dump is in attachment.

@anonymous-matomo-user
Copy link
Author

Attachment:
Dump20131115.sql

@mattab
Copy link
Member

mattab commented Dec 8, 2013

There are no pages views imported in the database. I think it could be caused by some special Mysql setting. Could you try again to import the log file and then see the mysql warnings with: SHOW WARNINGS query

http://dev.mysql.com/doc/refman/5.0/en/show-warnings.html

@anonymous-matomo-user
Copy link
Author

Hi matt,
You're right, i make a fresh install of my MySQL server and the problem disappeared. I don't know why but it's ok. you can close this ticket.

Thx

@anonymous-matomo-user anonymous-matomo-user added this to the 2.0.3 - Piwik 2.0.3 milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. worksforme The issue cannot be reproduced and things work as intended.
Projects
None yet
Development

No branches or pull requests

2 participants