Improvements to search engine and social network detection #9174
Conversation
sgiehl
added
Enhancement
| { | ||
| $url = 'https://raw.githubusercontent.com/piwik/searchengine-and-social-definitions/master/SearchEngines.yml'; | ||
| $list = Http::sendHttpRequest($url, 30); | ||
| $searchEngines = SearchEngine::getInstance()->loadYmlData($list); |
There was a problem hiding this comment.
we should check that the downloaded files are valid, eg. array has more than 200 elements, before overwriting the list
|
Nice idea, looks good to me! Maybe this could help other people contribute to the list of Search engines and social websites. For repository name maybe we could use Adding to 2.15.1 as it looks safe to push in this release. |
|
The mentioned improvements are done, I've create piwik/searchengine-and-social-list and added it on packagist.com. I'll check if I can manage to handle backward compatibility for the mentioned event. |
sgiehl
force-pushed
the
searchengines_and_social_improvements
branch
from
62368f8
to
641d836
Compare
sgiehl
added
Needs Review
|
Should be fine now. |
… better readable yml format
sgiehl
force-pushed
the
searchengines_and_social_improvements
branch
from
5902236
to
b402f19
Compare
|
just rebased on master. So if everything is fine, guess we could merge it now |
…ents Improvements to search engine and social network detection
|
Looks good, a refactoring that improves the platform and makes code cleaner. hopefully it will increase contributions to our social & search engine lists 👍 |
Currently the detection definitions for search engines and social networks are stored in two "data files" containing big php arrays.
In order to make that more readable and easier to extend I've converted it to YAML files.
To make it possible to provide an automatic update for those definitions I've moved them to a separate repository (currently located in my account sgiehl/searchengine-and-social-definitions) and added a handling similar to the referrer-spam list.
I've also moved most of the code handling the search engine and social network detection to the
Referrerplugin, as it is only used there.NOTE: This changes will break compatibility of plugins using the
Referrer.addSearchEngineUrlsevent to define own search engines. (e.g. sgiehl/piwik-plugin-ReferrersManager)Any thoughts / comments?
Todo's (if we want to merge)
Referrer.addSearchEngineUrlsevent (if merged before 3.0)refs #8980