Do not track the data when a token_auth is set to the Tracking API but is not valid #7202

iquito · 2015-02-13T23:07:59Z

I am using the PiwikTracker class und submitting log entries in bulk to Piwik through a cron job. I noticed that since a Piwik upgrade in December (), setting the user IP does not work anymore. I am using code in the following fashion (simplified):

$t = new \PiwikTracker( 1, 'url-to-piwik');
$t->setTokenAuth( 'my-admin-md5-hash' );
$t->enableBulkTracking();

while(log entries in $info variable)
{
  $t->setVisitorId(substr(md5( $info['session_id'] ), 0, 16));
  $t->setIp( $info['user_ip'] );
  $t->setForceVisitDateTime( gmdate('Y-m-d H:i:s', $info['create_date']) );
  $t->doTrackPageView($page_title);
}

Instead of the IP I am setting explicitely with setIP, my server IP address is used. I checked PiwikTracker.php and did not find anything wrong there, PiwikTracker.php creates the following API url:

https://myserverdomain.com/piwik/piwik.php?idsite=1&rec=1&apiv=1&r=232237&cip=178.197.230.13&cdt=2015-02-13+22%3A24%3A58&_idts=1423868304&_idvc=0&res=360x559&cookie=1&cvar=%7B%223%22%3A%5B%22_pks%22%2C106460%5D%2C%224%22%3A%5B%22_pkn%22%2C%22Hei+Poa+Soin+Trad+Monoi+Roucou+Effet+Cuivre+100+Ml%22%5D%2C%225%22%3A%5B%22_pkc%22%2C%22%22%5D%7D&gt_ms=31&cid=1285616f562e29e1&url=https%3A%2F%2Fwww.myshop.ch%2Fp106460%2Fhei-poa-soin-trad-monoi-roucou-effet-cuivre-100-ml&urlref=https%3A%2F%2Fwww.google.ch

The IP address is contained as it should be, yet the Piwik API seems not to use it.

I am using the newest Piwik version (2.10.0), the error already occured in 2.9.1. and maybe further back.

The text was updated successfully, but these errors were encountered:

tsteur · 2015-02-14T01:19:12Z

Does the given auth token have admin permission for that site or super user permission? Is the token auth present in the original URL?

iquito · 2015-02-14T01:30:00Z

The auth token has admin permission, and $t->setForceVisitDateTime works perfectly fine, which is only possible with admin/super user permissions.

The auth token is contained as POST variable, not in the URLs, as I am doing bulk tracking - I guess I should have included a "$t->doBulkTrack();" at the end of my example to make it more clear. Everything within PiwikTracker.php looks fine, that is why I am concluding that the tracker API is ignoring the cip-parameter somehow.

iquito · 2015-02-14T01:31:16Z

Maybe this is just a bug with bulk tracking? It is probably easy to find out if one knows the piwik API code.

tsteur · 2015-02-14T22:32:28Z

I just tried to reproduce this via bulk tracking (via PiwikTracker) but worked fine for me. I also had a look at $t->setForceVisitDateTime and this doesn't necessarily need admin/super user permissions. We do accept this value if the forced time is not older than 4 hours. Can you maybe double check the token? And maybe check whether it is actually forwarded to the Tracking API?

I tested using the latest 2.11 beta version so it is also possible that it was fixed somehow but I doubt that. I tried it with a super user token and an admin token.

iquito · 2015-02-14T22:45:35Z

Where can I find out all the tokens which are valid? I never changed the tokens and my script used to work (and it did not change either), but maybe Piwik changed the tokens after the upgrade? That's the only explanation I can come up with if the problem is with the token.

And how can I completely disable the tracking API when used without a valid token? This seems like a huge security risk to me (and everybody can theoretically change the statistics, if they know the URL).

iquito · 2015-02-15T18:38:42Z

I checked for the API token when logged in to Piwik, changed it in my code, and the IPs are being set again when running my script. The problems started immediately after upgrading from 2.9.0 to 2.9.1 - I did not change the token or anything else about the user account, at that time I did not use the account, I only updated the Piwik version.

I still see huge problems with the current workings of Piwik in this regard:

Anybody can insert any kind of visit entries without a valid auth token, only the IP addresses cannot be changed. This is a huge security loophole which can be used to corrupt any kind of statistics and are an easy target for DDOS attacks. I searched all the Piwik options, but did not find any kind of option or possibility to limit this behavior.
If the auth token is changed for any reason, the statistics are incomplete and there is no indication anywhere about the problem. If inserting entries would just have stopped working, I could have corrected the problem 2 months ago, now I have 2 months worth of statistics where every single IP address is wrong. Can this be somehow changed, or can the visits of the past 2 months be deleted and re-inserted?

tsteur · 2015-02-15T19:26:50Z

Re 1) There is for example bulk_requests_require_authentication to always require auth for bulk requests which would kinda also solve your second point but only for bulk requests.

Re 2) I agree, I was thinking about this as well. If token is not valid we should not simply ignore the values but not insert the request at all or somehow differently make clear that there is an error. Ping @mattab

mattab · 2015-02-15T22:39:39Z

If token is not valid we should not simply ignore the values but not insert the request at all or somehow differently make clear that there is an error.

👍

Maybe this issue scope could be "Throw an error/exception when a token is set to the Tracking API but is not valid" ?

iquito · 2015-02-15T23:00:10Z

From my perspective the following changes would be great:

bulk_requests_require_authentication should be set to "true" as default. I don't think there is a valid use-case to bulk inserting without authentication, the risks in terms of security are too great. (Any third party could insert a huge amount of entries in a short time and overload a Piwik installation easily)
Make a new configuration option to only allow access to the whole tracking API with a valid auth token. I suspect the Javascript code is the reason why the tracking API is "open" by default, but when using a different system than the Javascript code there is no need for the tracking API to accept any requests without a valid auth token, which can be kept secret. If an option exists to restrict the tracking API, this would enable Piwik to be run in a very secure manner.
PiwikTracker.php should throw an exception if the auth token is not valid. This would immediately alert somebody using the class about an error with the auth token.

tsteur · 2015-02-15T23:20:53Z

bulk_requests_require_authentication should be set to "true" as default. I don't think there is a valid use-case to bulk inserting without authentication, the risks in terms of security are too great.

There is a valid reason for this since bulk tracking is eg used by some clients by default eg the iOS tracker and we can't require a token here. If someone wants to overload your Piwik installation, one can do it without bulk requests as well and maybe even easier by just sending many single tracking requests. As you said the javascript tracking code is the reason for this. An option to only allow the tracking with code sounds good to me. PiwikTracker.php checking the tracking response code and throwing an exception sounds good to me as well.

iquito · 2015-02-16T02:28:36Z

Ok, for a default setting there are always different possibilities according to preferences. Personally, I would set it to true by default, but changing it now would break existing applications, which is not great. With a new option to restrict tracking to auth tokens and the exceptions in PiwikTracker when invalid auth tokens are used, my problem or anything like it should not occur anymore.

urru · 2015-03-05T22:04:29Z

I had this issue with tracking requests. Auth_token changed autonomously with an upgrade and subsequently the cip parameter of all tracking requests was replaced with the server ip. I too now have months of invalid ip addresses, which for an analytics system means this is quite important.

mattab · 2015-03-05T23:40:42Z

Piwik does not change token_auth automatically - however token is changed whenever the password is changed.

I will rename this issue to reduce the scope to Throw an error/exception when a token is set to the Tracking API but is not valid

(implementation note: i think the only way to do this will be that Tracking API returns an error eg http code 403 when the token is invalid.)

iquito · 2015-03-05T23:55:47Z

My token_auth definitely changed when I upgraded Piwik - I did not change my password or my account in any way. That is how my problems with the wrong IPs started.

mattab · 2015-04-07T06:12:28Z

we can't start throwing errors as it could stop tracking data for some Piwik setup: it's an API BC break (it would start showing errors when it used to track data). Moving to Mid term for now

tsteur · 2015-04-07T06:30:13Z

It wouldn't track anyway or not? Or maybe it would track wrong data. I'm not sure but this sounds actually useful. How would other people notice that the tracking doesn't work properly anymore? This also relates to #7550

IMO if a token auth is set, and it is not valid, it should not track anything is it is potentially malicious

mattab · 2015-04-08T02:48:11Z

It wouldn't track anyway or not? Or maybe it would track wrong data.

yes it would track with possibly wrong data...

IMO if a token auth is set, and it is not valid, it should not track anything is it is potentially malicious

+1 - this change would make sense and help users spot an issue when they don't see data anymore

Notes:

the main concern if we throw an error is that if we would also need to make sure such error request would not break the Bulk request, as it could break Logs import process. we'd need to add test case
we should add a note to the FAQ: No data in reports and explain that using the wrong token_auth in API requests could lead to data not tracked.

mattab · 2016-12-26T04:40:51Z

if a token auth is set, and it is not valid, it should not track anything is it is potentially malicious

This was fixed in Piwik 3.0.0 in #10899 #10890

mattab added the Task Indicates an issue is neither a feature nor a bug and it's purely a "technical" change. label Feb 15, 2015

mattab added this to the Short term milestone Feb 15, 2015

mattab changed the title ~~IP not set when sent via Tracking HTTP API~~ Throw an error/exception when a token is set to the Tracking API but is not valid Mar 5, 2015

mattab modified the milestones: Mid term, Short term Apr 7, 2015

mattab modified the milestones: Short term, Mid term Apr 7, 2015

mattab changed the title ~~Throw an error/exception when a token is set to the Tracking API but is not valid~~ Do not track the data when a token_auth is set to the Tracking API but is not valid Apr 8, 2015

tsteur mentioned this issue Aug 7, 2015

when using LDAP authentication, tracking api fails to verify token_auth #8509

Closed

Joey3000 mentioned this issue Sep 18, 2015

Restricting token_auth access #8816

Closed

tsteur mentioned this issue Nov 26, 2015

Make sure local time has valid time format #9280

Merged

mattab closed this as completed Dec 26, 2016

mattab added the duplicate For issues that already existed in our issue tracker and were reported previously. label Dec 26, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not track the data when a token_auth is set to the Tracking API but is not valid #7202

Do not track the data when a token_auth is set to the Tracking API but is not valid #7202

iquito commented Feb 13, 2015

tsteur commented Feb 14, 2015

iquito commented Feb 14, 2015

iquito commented Feb 14, 2015

tsteur commented Feb 14, 2015

iquito commented Feb 14, 2015

iquito commented Feb 15, 2015

tsteur commented Feb 15, 2015

mattab commented Feb 15, 2015

iquito commented Feb 15, 2015

tsteur commented Feb 15, 2015

iquito commented Feb 16, 2015

urru commented Mar 5, 2015

mattab commented Mar 5, 2015

iquito commented Mar 5, 2015

mattab commented Apr 7, 2015

tsteur commented Apr 7, 2015

mattab commented Apr 8, 2015

mattab commented Dec 26, 2016

Do not track the data when a token_auth is set to the Tracking API but is not valid #7202

Do not track the data when a token_auth is set to the Tracking API but is not valid #7202

Comments

iquito commented Feb 13, 2015

tsteur commented Feb 14, 2015

iquito commented Feb 14, 2015

iquito commented Feb 14, 2015

tsteur commented Feb 14, 2015

iquito commented Feb 14, 2015

iquito commented Feb 15, 2015

tsteur commented Feb 15, 2015

mattab commented Feb 15, 2015

iquito commented Feb 15, 2015

tsteur commented Feb 15, 2015

iquito commented Feb 16, 2015

urru commented Mar 5, 2015

mattab commented Mar 5, 2015

iquito commented Mar 5, 2015

mattab commented Apr 7, 2015

tsteur commented Apr 7, 2015

mattab commented Apr 8, 2015

mattab commented Dec 26, 2016