Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better Device detection (mobile, TVs, consoles, etc.): New "Devices" report? #3612

Closed
mattab opened this issue Dec 14, 2012 · 14 comments
Closed
Assignees
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Milestone

Comments

@mattab
Copy link
Member

mattab commented Dec 14, 2012

Piwik should provide a more clear set of reports regarding Devices, Operating systems, browsers, significantly enhancing the analytics insights on the visitors browsing experience.

This ticket is about adding a couple new report and fixing related issues at the same time. Proposal:

  • Add New Device type report comparing generic families of devices: Computer, Mobile, Tablet, TV, Console, Portable console.
    • It would be displayed between "resolutions" and "mobile vs dekstop" report.
    • This would be a superset of the existing reports "Operating systems" which does already a simple "device" aggregation (especially reports mobile devices separately since they run a different OS by design). I propose to add a "related report" link that will not load the report in ajax, but will instead link to #devices hash tag in the page below to load the 'Device type report'
  • Add new Manufacturer" report that would grep the user agent for known brands: Samsung, Apple, HTC, Motorola, Amazon, etc.
    • we need to find the full list somewhere
    • and their favicons.
  • Add new Manufacturer+ Model report reporting eg. Nokia N80
  • Modify Browser version report to segment by Opera Mobile VS Opera #2807
  • Modify Operating systems report

Also see a feature request: Detecting TVs #3505

While we work on this, please let's try to implement this as well: #3157 Add a new visitor segment attribute "isMobile" (including tablet)
Open for discussion...

@mattab
Copy link
Member Author

mattab commented Dec 20, 2012

Note: this will require some changes in the Schema of log_visit and log_conversion so this is definitely not trivial change. Before starting work on this one please get in touch to discuss specs.

@anonymous-matomo-user
Copy link

I would like to recommend a separate tab between Referers and Goals called "Mobile". The mobile universe is such a critical part of digital online strategies that it should be separate in this way. Most companies like to segment their mobile data so capturing it and presenting this way will feel very natural. Once there a sub set of tabs would then list the Following in Order of importance:

Device (Specific Device name, (Note would be great to perform regex like filtering to allow grouping of devices by families)
Device Type (generic device family type as noted earlier(Computer, Mobile, Tablet, TV, Console, Portable console), excellent for marketing analysis)
Manufacturer
Operating System
Browser Version
Screen Size (HxW)
Video Support (Various types of video formats if available)
Net Protocal

Standard Metrics PVs UVs and Geo Location (using lat and long)

The report "Mobile VS Desktop" Devices, Device type and manufacturer could be the initial Widgets under the New Mobile tab. If only one report is allowed then use Mobile VS Desktop.

Sidenote: Aps a component of the Mobile universe should really be treated as their own site from the master site pop up. Companies like to compare and/or total their desktop sites to mobile apps so it would be great if the all sites report allowed the basic PVs and UVs to display together.

@anonymous-matomo-user
Copy link

One thing that could help to speed up the range of devices/OS/browsers that Piwik can recognize as specific ones, is some kind of "User Agent Logger" that could be turned on/off in the configuration tab of every Piwik installation.

It would save all the User Agent Data from unknown User Agents as a list plus an optional text field where the Administrator could add some notes per User Agent (e.g. a suggestion of the kind of device this agent should berecognized).

As a manual task, the Admin could "upload" this list from time to time to some kind of master list here, which would be used continuously to maintain the existing device/OS/Browser database. The upload itself could be done automatically by pressing a button, but the upload task should not be automated to avoid giving Piwik an image of automatically uploading user data somewhere.

@mattab
Copy link
Member Author

mattab commented Jan 4, 2013

@old_bill we won't do anything like that, but instead will use a library that mostly works and that we can update later on when new devices are released. Updating the list will happen when you update Piwik ~ every month or two.

If you're interested in this ticket and can sponsor please get in touch NOW with us at: http://piwik.org/consulting/#contact-consultant

@anonymous-matomo-user
Copy link

It would be great if you could adjust current browser/OS reports to look like those in this plugin (I know it's flash based, so it can't be just re-used)

#1558
http://issues.piwik.org/attachments/1558/screenshot.png

I find this much more informational than current pie charts.

@anonymous-matomo-user
Copy link

Hi, does this mean there's no easy way to work out what % of our traffic is mobile/tablet?

We're considering creating more responsive designs and we're trying to justify the outlay.

@mattab
Copy link
Member Author

mattab commented May 14, 2013

In 83904c1: Adding user agent to the newVisitorInformation tracker hook
refs #3612

@mattab
Copy link
Member Author

mattab commented May 17, 2013

In 7a3e6f4: Refs #3612 Adding new icons for browsers/os detected by upcoming advanced device detection

@mattab
Copy link
Member Author

mattab commented May 24, 2013

In 6804733: Adding new DevicesDetection plugin, developed by Clearcode and sponsored by a client of Piwik Professional Services. A beautiful work of engineering, all released under GNU/GPL license!

Fixes #3612

  • when enabled, the plugin will add a new submenu: Visitors> Devices
  • the new devices report contains NEW reports:
    • Much improved Device type (tracking 'car browser', 'console', 'desktop', 'feature phone', 'smartphone', 'tablet', 'tv')
    • Device brand (tracking more than 100 brands such as Nokia, Nintendo, Lenovo or Apple.
    • Device model (tracking hundreds of phone/console models)
    • Operating System versions (tracking 70 variations of operating systems including Ubuntu vs Kubuntu vs Debian vs Lubuntu vs Xubuntu)
    • Operating System families (Android vs Google TV vs Windows vs Windows mobile vs Mobile gaming consoles)
    • Browser versions
    • Browser families

Refs matomo-org/device-detector#3505 There is some basic TV detection included and maybe you can help contribute better detections (see the .yml data files containing the regular expressions in YAML format)

HOW DOES IT WORK

This is quite beautiful system. It is a plugin that disabled by default. when enabled, it will create additional columns in the DB. Also at tracking, it will look at the user agent, and try to match it against one browser we know in the databases. The databases of user agent matching are composed by 3 YML files, parsed by spyc.php into php array.

These 3 YML took dozens of hours of work and testing with dozens of mobile phones and devices for accuracy. We are happy with the result as they should cover > 80% of the devices commonly used. We hope the community will help us build up these YML files and make them better, so we can track accurately 90% or 95% of requests.

The performance overhead is pretty small, but parsing the YML files + running dozens of regex will add some overhead. This is why it is still disabled by default. We will think about how to integrate it in core, in the next few months.

Please let me know if you find any problem with this new awesome code!

@mattab
Copy link
Member Author

mattab commented May 24, 2013

In 4f4d377: Refs #3612

  • making some tests pass == fixing bugs

@mattab
Copy link
Member Author

mattab commented May 24, 2013

In 58c88c1: Refs #3612

  • making tests pass!
  • enabling DevicesDetection tests, even though DevicesDetection plugin is disabled by default.

@mattab
Copy link
Member Author

mattab commented May 25, 2013

In 771239a: Adding Chrome mobile icon (ideally we need a unit test to check we have icons for all browsers/OS referenced in the UserAgentParserEnhanced)
Refs #3612

@mattab
Copy link
Member Author

mattab commented May 30, 2013

See also: #3961 New segment: deviceType to let you segment visitors by: desktop, smartphone, tablet, feature phone, console, tv, car browser

@mattab
Copy link
Member Author

mattab commented Feb 4, 2014

See follow up ticket: #3962

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement For new feature suggestions that enhance Matomo's capabilities or add a new report, new API etc. Major Indicates the severity or impact or benefit of an issue is much higher than normal but not critical.
Projects
None yet
Development

No branches or pull requests

2 participants