@Findus23 opened this Issue on June 26th 2020 Member

(I thought I wrote about this before, but couldn't find an issue)

The Chrome team is working on a large change that plans to deprecate the user-agent string.
The current(?) Chrome 83 already freezes the string and Chrome 85 will unify it to one common string for all desktop and one for all mobile users.

As a replacement the client hints standard is proposed: https://github.com/WICG/ua-client-hints

This allows to get similar data about a device already separated by things like brand, browser, os, etc. (meaning device-detector would not be needed anymore).

There is both an HTTP header (that only returns non-vague results if the server requests them) and a JS API for accessing the data.

Things that might complicate this:

the main UA value is not a string, but a list/set

Chrome"; v="73", "ChromiumBasedBrowser"; v="60", "Chromium"; v="73" (and the corresponding dictionary from JS) is a valid response and browser vendors are encouraged to "randomly [include] additional, intentionally incorrect, comma-separated entries with arbitrary ordering".


It seems like at the moment only Chromium-based browsers are working on implementing this standard, but as it seems like these are by far the majority now, that doesn't really matter.

I added this to 4.0 RC for now as it impacts every Matomo user as the most popular user agent will display incorrect results.

Links:
https://groups.google.com/a/chromium.org/forum/#!msg/blink-dev/-2JIRNMWJ7s/yHe4tQNLCgAJ
https://www.chromestatus.com/feature/5995832180473856
https://github.com/WICG/ua-client-hints
https://wicg.github.io/client-hints-infrastructure/
https://github.com/mozilla/standards-positions/issues/202#issuecomment-558294095 (Position of the firefox team on this standard)

@tsteur commented on June 28th 2020 Member

BTW @Findus23 just tested using latest Chrome 83 and Chromium 86. Both of them did not send any of the new headers, neither is the new navigator property set to read this through JS, nor did it look like the user agent was frozen. Wonder if these things were delayed eg due to covid 19

@tsteur commented on June 28th 2020 Member
@tsteur commented on July 17th 2020 Member

checked again and seems there's still no update

@tsteur commented on September 3rd 2020 Member

This feature is still in development in Chrome

@sgiehl commented on October 1st 2020 Member

It's at least meanwhile possible to read the values after enabling them in Chrome > 85:
See https://user-agent-client-hints.glitch.me/

@tsteur commented on October 1st 2020 Member

After enabling the experimental flag this worked for me. It was setting by default eg these headers along the tracking request

sec-ch-ua: "\\Not;A\"Brand";v="99", "Google Chrome";v="85", "Chromium";v="85"
sec-ch-ua-mobile: ?0

Platform headers would not be sent. Be great to focus on the other issues first in the RC milestone and towards the end on this one maybe.

Is the plan that we support these new headers in the device detector @sgiehl ? Or maybe we instead make eventually use of the JS api navigator.userAgentData and send this information along the tracking request once we are able to retrieve it so over time we in a few years we might be able to no longer use device detector which be great (as it is taking too much work). Of course depends if other browsers will implement this as well.

And it will be up to Matomo/website owner to send the correct headers so we receive the needed information?

@sgiehl commented on October 2nd 2020 Member

I think I wouldn't add that to device detector actually. We should handle that within Matomo...

@tsteur commented on November 18th 2020 Member

@sgiehl @mattab I'll move this one to Matomo 4.1

@tsteur commented on November 18th 2020 Member

@sgiehl @Findus23 it's not clear to me when they will start freezing user agent. Is it though already worth looking into this what we need to do to make this work?

@sgiehl commented on November 18th 2020 Member

I already had a look on that a few days ago. It's possible to access those client hints using javascript like this:

navigator.userAgentData.getHighEntropyValues(["model", "platform", "platformVersion", "uaFullVersion"])
   .then(function(ua) {
         var model = ua.model;
         var os = ua.platform;
         var osVersion = ua.platformVersion;
         var browserVersion = ua.uaFullVersion;
});

As you can see it's accessible using a promise only, which will make it a bit more difficult to integrate into piwik.js I think. Additionally I did not yet find any useful information which values the variables can have. I only see the values my Chrome has...

What we need to do on Matomo tracking side is to identify the short codes for browser, os and brand based on the given values. Not sure how easy that will be. But guess we need to see that as soon as some more details on the values are known.

@Findus23 commented on November 18th 2020 Member

I honestly can't find any news on this that is newer than my initial post. It seems like Google has posponed the planned changes. The only news are that Chrome now has an experimental flag for enabling the client hints.

And I think if the proposal is actually implemented as intended by Google¹ then I am not sure if it is useful to record the output in Matomo as it will intentionally give randomized incorrect results. https://wicg.github.io/ua-client-hints/#grease

https://github.com/w3ctag/design-reviews/issues/467 shows a bit more discussion about the topic of freezing ua strings (and the least helpful "graph" I have ever seen).


¹ This proposal is still only created and discussed by Google (and Microsoft Edge) developers, no non-chromium-browser engines have any plans on implementing it.

@Findus23 commented on November 18th 2020 Member

> Will the delay also include freezing the UA string (i.e. will the current UA string be up-to-date until some change in 2021).
That is correct

from https://groups.google.com/a/chromium.org/g/blink-dev/c/-2JIRNMWJ7s/m/yHe4tQNLCgAJ

@tsteur commented on November 18th 2020 Member

👍 let's maybe wait until they announce it again (which I think they will).

@ampaze commented on April 26th 2021

Waiting for the Browsers to start freezing the User Agent before implementing Client Hints is really asking for trouble...

Client Hints are enabled by default in Edge 90 and Chrome 90, it won't be that long before User Agent freezing is here.

@Findus23 commented on April 26th 2021 Member

It would be great if Client Hints could become a browser standard (both https://wicg.github.io/ua-client-hints/ and https://datatracker.ietf.org/doc/html/rfc8942 seem to be a draft and experimental) and be implemented by more than one browser (engine). Also I don't see any specific information that User Agent freezing is no longer delayed indefinitly (https://bugs.chromium.org/p/chromium/issues/detail?id=955620).

But then again it seems like that's not how the Google Monopoly on browser standards works at the moment, so maybe it might be useful to implement this one day before Google suddenly adds it to all browsers like it did with FLoC.

@ampaze commented on April 26th 2021

But then again it seems like that's not how the Google Monopoly on browser standards works at the moment, so maybe it might be useful to implement this one day before Google suddenly adds it to all browsers like it did with FLoC.

That is most likely how it will work out, even if other browsers don't follow. Chrome alone has enough users, that it might be wise to be prepared for a potential UA-Freeze, even if it will be postponed by Google for a while.

@tsteur commented on April 27th 2021 Member

Moved it back to 4.3 milestone 👍

@justinvelluppillai commented on July 16th 2021 Contributor

I've revisited this with @tsteur and it looks like chrome hasn't frozen UA strings yet. We could extract browser version from userAgentData fairly easily. It will require some work to normalise OS and browser information for matomo.

Mozilla has also updated their view on client hints to "harmful" https://github.com/mozilla/standards-positions/issues/552

We will revisit this later.

Powered by GitHub Issue Mirror