@anonymous-matomo-user opened this Issue on April 30th 2011

Hi, I use Yottaa to monitor my websites, but their bots appear in the statistics.
While they don't provide a list of ip's ("because our bots come from all over the world and there's no set block of addresses you could use" ), they suggest to exclude the bits from their user-agent "YottaaMonitor".
It would be nice to have a feature to exclude those bots.


@diosmosis commented on November 22nd 2012 Member

Attachment: Patch for this issue.

@robocoder commented on April 30th 2011 Contributor

I've quick-fixed the Yottaa filter in #2376.

@anonymous-matomo-user commented on April 30th 2011

So I close this ticket, thanks !

@robocoder commented on April 30th 2011 Contributor

Well, I'll keep this ticket open. I still like the idea of having this as a general feature since the image tracker has more exposure to bot traffic.

@anonymous-matomo-user commented on July 1st 2011

I politely request this feature, too, as any web site using a monitoring systems (WatchMouse, Pingdom) that emulate a real browser can have their statistics skewed. Those systems tend to hit the site once every minute, from different IPs and different parts of the world.

It would be great if there was an option/button on the Visitor: Log, Providers, and Settings pages to exclude something based on the visitor's or providers IP/host, and, most importantly, the User Agent.

Many thanks,

@mattab commented on September 17th 2011 Member
  • Let's add a new field in Websites > General settings to exclude a visit based on user agent, across all websites.
  • Note: cache the user agents to exclude in the cache/tracker/ file via the hook Common.fetchWebsiteAttributes
  • Visit should be excluded as long as it "contains" the user agent (not exact match)
@anonymous-matomo-user commented on April 10th 2012

I would like to add my name to the list here, this would be a really useful feature as currently I can't really enable monitoring tools on our websites.

Obviously excluding the IPs by hand is possible, but not really desirable as the IPs are subject to change when servers are added/removed/moved.

@anonymous-matomo-user commented on October 24th 2012

I would like to add my vote to this as well.
We're also using a monitoring service with a certain User Agent. Since monitoring is important to know if the site is running and statistics (Piwik) are important to know who's visiting, it seems to me this is an important request.
Without this option, the statistics are completely useless when also using a monitoring service (which has many IP address which also change regularly).


@mattab commented on October 25th 2012 Member

Let's work on this feature which will be an important improvement :)

See spec in: #2375


@anonymous-matomo-user commented on October 29th 2012

Thanks, matt! This will be really useful.

@anonymous-matomo-user commented on November 14th 2012

Hello Matt,

I don't want to sound annoying or anything, but could you please move this ticket to (for example) release 1.10? I see it's not yet included in 1.9.2 and I don't see it in the list for 1.10.


@diosmosis commented on November 22nd 2012 Member

Hey matt, I attached a patch that fixes this issue. Let me know if it's good to commit.

@mattab commented on November 22nd 2012 Member

@capedfuzz Very nice!

Code review:

  • NoVisitTest.php should not change position of the updateSite setting the excluded IP or the test is for user agent exclude is not executed
  • we are already at b7 so please set to b8 or b9
  • setGlobalExcludedUserAgents() should require super user: otherwise any admin user in a shared piwik server, could set to exclude "mozilla,msie,chrome,opera" and cause some trouble
  • getExcludedUserAgentsGlobal() should require at least some admin access.. but even
  • UX KISS: 99% of users will never specify a "per website user agent". In fact it is not in the specification for this ticket ;) I thought that only "global user agent exclude" would be enough... but since you've built it, and that I can think of some use cases where this is useful, then
    • However since most people wil never use the feature, we should hide the column on the webvsite listing by default. Can you please, add a checkbox in the "super user User Agents" section, that reads "Enable custom User Agents to exclude for each website" (or similar). the box is unchecked by default (and the column "User agents to exclude" hidden from the table). When users click and press save, the page reloads and the columns is displayed.
    • NOTE: when the preference is set from "Enable custom agents for each site" from checked to unchecked, then all excluded_user_agents columns should be deleted to later avoid hard to find tracking bugs.
    • implementation; to keep code simple, I would recommend to simply hide the column using jquery, this way the code is unchanged appart from getting/setting this new user preference?
@diosmosis commented on November 24th 2012 Member

(In [7535]) Fixes #2375, added ability to exclude visits from tracking based on global & site-specific blacklists user agent substrings. Also fixed bug in website management page that broke website editing/creation.

This Issue was closed on December 17th 2012
Powered by GitHub Issue Mirror