Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ActiveProvider plugin: Enhanced Provider Report uses Public Suffix List for second and third level domains #558

Closed
anonymous-matomo-user opened this issue Feb 18, 2009 · 9 comments
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. worksforme The issue cannot be reproduced and things work as intended.

Comments

@anonymous-matomo-user
Copy link

  1. ActiveProvider

The ActiveProvider plugin enhances the Provider report by using the cross-vendor supported Public Suffix list — hosted by the Mozilla Foundation at http://publicsuffix.org/ — to more accurately detect second and third level domains for a given top level domain.

The core Provider plugin does not recognize many public second and third level domains. For example, let’s say, you get a visit from $customer.$provider.co.uk. This is correctly recognized as $provider. But if you get a visit from $customer.$provider.edu.hk, it is recognized as edu.hk, not as $provider.

  1. Performance
    - added memory footprint approx. 960K
    - execution overhead (including load time) approx. 50ms on an (ancient) Athlon 1.4 GHz processor (test box), without APC
  2. Requirements
    - Piwik 0.5.5 (or above)
  3. To install
    - Extract the attached .zip file in the plugins folder.
    - Activate the plugin from the Settings | Plugins tab.
  4. For support
    - Send me a tweet [@vipsoft](http://twitter.com/vipsoft)
    - DO NOT CREATE A NEW TICKET HERE

Keywords: third-party-plugin

@robocoder
Copy link
Contributor

There’s quite a bit more overhead to using the [Public Suffix List](http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/src/effective_tld_names.dat?raw=1), both in terms of administration (e.g., keeping up to date with changes to the list) and performance (compared to the current method in plugins/Provider/Provider.php).

```
private function getCleanHostname($hostname)
{
$extToExclude = array(
‘com’, ‘net’, ‘org’, ‘co’
);
```

Attached patch uses Toby Inkster’s GPL’d code, found at http://tobyinkster.co.uk/blog/2007/07/19/php-domain-class/

@mattab
Copy link
Member

mattab commented Feb 25, 2009

overhead of using this class and the .dat list is quite big, and we really want to keep execution of piwik.php optimal. It would be disabled by default. However i think the best in this case might be to provide a plugin to do this advanced check. vipsoft, maybe you could add a hook in the getCleanHostname that would load the hostname from a plugin if hook is listened, or default to the normal simple algorithm. Creating the plugin from your existing patch will be easy. Does it make sense?

@robocoder
Copy link
Contributor

I should have done that the first time. ;)

@robocoder
Copy link
Contributor

(In [1750]) refs #558 - add Provider.getCleanHostname hook

@robocoder
Copy link
Contributor

(In [1753]) fixes #558 - plugin to use Public Suffix List to enhance Provider report; the register-domain-libs contains a PHP data structure to represent the contents of effective_tld_names.dat -- this loads and executes much faster (and can be opcode cached) than the implementation using Domain.class.php

@robocoder
Copy link
Contributor

(In [refs #558, revert 1753), remove PublicSuffix plugin from core per matt's review

@robocoder
Copy link
Contributor

(In [refs #558, revert 1753), move these lines back to top per matt's review

@robocoder
Copy link
Contributor

Attachment: 2010.01.08 update of Public Suffix List
ActiveProvider.zip

@robocoder
Copy link
Contributor

Attachment: 2011-03-01 update of Public Suffix List
ActiveProvider-1.2.1.zip

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. worksforme The issue cannot be reproduced and things work as intended.
Projects
None yet
Development

No branches or pull requests

3 participants