Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking API: Handle when older visitor/page views data is imported after the newer data... #3653

Closed
anonymous-matomo-user opened this issue Jan 7, 2013 · 4 comments
Labels
Bug For errors / faults / flaws / inconsistencies etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.

Comments

@anonymous-matomo-user
Copy link

I have the IIS logs of load balance servers like this structure:

Server 1: log1/
u_ex121119.log
u_ex121120.log
u_ex121121.log

Server 2: log2/
u_ex121119.log
u_ex121120.log
u_ex121121.log

Server 3: log3/
u_ex121119.log
u_ex121120.log
u_ex121121.log

When import the data from the log to the Piwik, I get the errors like this:

Error when connecting to Piwik: <urlopen error None>

I've debugged the exceptions, it showed that there were two variables were out of range:

$visitTotalTime and $timeSpentRefererAction

they were all negative numbers:


Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'visit_total_time' at row 1

Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_spent_ref_action' at row 1
@anonymous-matomo-user
Copy link
Author

BTW, The code file is:

/core/Tracker/Visit.php

@anonymous-matomo-user
Copy link
Author

  1. Throw exception when the time is invalid
$valuesToUpdate['visit_total_time'] = $visitTotalTime + 1;

/** FIXES START */
if($valuesToUpdate['visit_total_time'] < 0) throw new Piwik_Tracker_Time_Invalid();
/** FIXES END */

// Goal conversion
if($visitIsConverted)
  1. Throw exception when the time is invalid
// Will be updated in cookie
$timeSpentRefererAction = $this->getCurrentTimestamp() - $this->visitorInfo['visit_last_action_time'];
if($timeSpentRefererAction > Piwik_Config::getInstance()->Tracker['visit_standard_length'])
{
    $timeSpentRefererAction = 0;
}

/** FIXES START */
if($timeSpentRefererAction < 0) throw new Piwik_Tracker_Time_Invalid();
/** FIXES END */

$this->visitorInfo['time_spent_ref_action'] = $timeSpentRefererAction;
  1. Create the new exception:
class Piwik_Tracker_Time_Invalid extends Exception
{
    function __construct($message = 'Invalid timestamp', $code = -1)
    {
        parent::__construct($message, $code);
    }
}

@anonymous-matomo-user
Copy link
Author

The problem will be fixed when I merge the logs and sort the visit information by time, the code above will prevent the error SQL executions.

BUT THE ISSUE STILL EXISTS, IS THAT POSSIBLE TO IMPORT THE DATA SUCCESSFULLY WITH RANDOM TIMESTAMPS?

For example, import a same-date-log from different servers:

Visitor1, Server1: 10:00:00
Visitor1, Server2: 10:30:00
Visitor1, Server3: 10:10:00 - This will be an error in the Piwik for now.

THANKS.

@mattab
Copy link
Member

mattab commented Jan 18, 2013

Thanks for the report and patch! I won't apply as it is because it should not be user facing and can only be possible with token_auth auth and overriding cdt.

It's not possible import logs in random orders. we could implement it but it would be a mess and is not on our busy roadmap.

@anonymous-matomo-user anonymous-matomo-user added this to the Future releases milestone Jul 8, 2014
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug For errors / faults / flaws / inconsistencies etc. wontfix If you can reproduce this issue, please reopen the issue or create a new one describing it.
Projects
None yet
Development

No branches or pull requests

2 participants