Hi @miketaylr!

The review in issue #467 was previously closed, but I’m requesting a new review now that Chrome is interested in pursuing this plan again.

Have there been any substantive changes since our last review? If so, could you give us a summary of them, so we know what to concentrate on? If not, I'm not sure what there is to do here.

Heya @hober!

Yeah, sorry. It probably would have been useful for me to do so as I filed this 🙈 . The most substantial changes can be summed up as:

• A more fleshed out spec exists for User-Agent Client Hints
• The original "UA Freezing" ideas proposed that all non-mobile browsers should report as Windows 10 devices. We've since changed our thinking on this, for reasons articulated in the blink-dev intent to prototype & ship thread. Instead, our current thinking is to keep the platform exposed in the User-Agent string, for compatibility and accessibility reasons (sadly many sites rely on sniffing for platform to handle keyboard shortcut differences between OSes).

edit: perhaps more usefully, these are the major updates since the original intent filed by @yoavweiss, and the ones captured in the Blink Intent to Prototype & Ship:

1. Sec-CH-UA-Bitness: adds a new high-entropy hint to expose the OS bitness, which may be combined with Sec-CH-UA-Arch to provide optimized binaries for download, for example.
2. Make Sec-CH-UA-Platform a low-entropy hint: OS is passively observable at the TCP level anyways, so we plan to change this to be low-entropy and send as a default header (similar to Sec-CH-UA and Sec-CH-UA-Mobile).
3. Include low-entropy hints by default in UADataValues (returned by getHighEntropyValues()). If a hint moves from high to low-entropy, this future proofs any code relying on it.
4. Add a toJSON method to NavigatorUAData’s IDL. Technically a bugfix, but it is an API change (instead of returning {}, JSON.stringify(navigator.userAgentData)) will now be useful)

In addition, we've merged in 67 PRs to the spec since #467 was first filed (some more substantial than others).

An early question - where does UA Client Hints go after WICG? Is the intention to bring this into the Web Apps working group, for example? It's not clear to me what the trajectory for this is?

I think they are more of a Web Perf WG thing - hi @yoavweiss!

Having said that, we can take them in WebApps WG, but we would need to add it as a thing to our charter. Our charter is up for renewal, so it's a good time to add. However, we'd like to make sure there is a support from a second implementer and no objection from a third.

The mental model I had was that each feature that relies on CH could go to the most relevant WG, while the infrastructure could go to WebPerfWG, or ideally integrated directly into Fetch and HTML. For UA-CH specifically, WebApps seems appropriate.

In terms of support or lack thereof, I'll note that Mozilla considered it non-harmful and @othermaciej previously said he's personally "mildly positive" on the direction with a few reservations, but since had at least one objection. I'm hoping these reservations and objections are things that can be worked out in the WG, rather than ones that would block adoption.

As noted in today's call we'll try to prioritize getting back to you with some feedback on the platform issue.

Sent https://github.com/w3c/webappswg/pull/54 to WebAppsWG to add to charter.

@miketaylr can you give us a bit more info on how other browsers are being engaged on this issue? I see some info about other engines but is there there active engagement?

(pardon the delay in response, in the middle of a move)

On UA Reduction:

Mozilla has taken independent steps to begin freezing information in the User-Agent string (see [1][2]), and is positive on the concept in general (see [3]). Apple has also frozen (or capped) most information in the Safari User-Agent string (see [4][5]). We would like to document how browsers have already taken these steps in the WHATWG compat standard (to start with, anyways - it may make sense for this to ultimately live in another spec). We met with Mozilla a few months back to discuss the issue of freezing the macOS version number (we invited Apple, but nobody was able to attend due to short notice). See https://www.otsukare.info/2021/03/02/capping-user-agent-string for minutes.

Personally, I would welcome participation from any browser vendor or interested party to help here: https://github.com/whatwg/compat/issues?q=is%3Aissue+is%3Aopen+%5Buastring%5D

On UA-CH:

Mozilla is neither positive nor negative on User-Agent Client Hints ("non-harmful"). We don't have a formal position from Apple one way or the other, but @othermaciej has filed a lot of high-quality bugs on the spec (only 1 out of 16 remain unresolved, see [6]). Edge has stated that they are supportive of User-Agent Client Hints (see [7]), and have also filed some high-quality bugs on the spec (two of which I hope to address soon, see [8]).

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1679929 [2] https://bugzilla.mozilla.org/show_bug.cgi?id=1693295 [3] https://github.com/mozilla/standards-positions/issues/202#issuecomment-558294095 [4] https://webkit.org/blog/8042/release-notes-for-safari-technology-preview-46/ [5] https://twitter.com/__jakub_g/status/1398324307814752256 [6] https://github.com/WICG/ua-client-hints/issues?q=is%3Aissue+author%3Aothermaciej [7] https://twitter.com/_scottlow/status/1206831008261132289 [8] https://github.com/WICG/ua-client-hints/issues?q=is%3Aissue+author%3Aerik-anderson+

We've taken note of Mozilla's recent feedback. It looks like information is coming from implementers which would call for more thorough review. @miketaylr do you have any response to Mozilla's position?

Hi @torgo, I responded yesterday to some of @hsivonen's comments at https://github.com/mozilla/standards-positions/issues/552#issuecomment-879162601, as they were based on outdated information (or simple misunderstandings perhaps).

I don't fully understand what "Harmful" means in the context of his post, as the info is already available in the User-Agent header. For example, for hints such as Sec-CH-UA-Model (which Firefox does not expose in its Android mobile browser UA strings), the spec explicitly states:

User agents MAY return the empty string or a fictitious value for full version, platform architecture, platform bitness or model, for privacy, compatibility, or other reasons.

If a browser were to implement UA-CH, didn't want to expose Model, or Full Version, etc., it has the ability to respond with the empty string and be conforming - either because some privacy policy or user setting was being enforced (Privacy Budget, Firefox's ETP Strict Mode, Firefox's "Resist Fingerprinting" mode, etc.) - or just because a vendor decides that should be default value.

Perhaps Henri can explain what he means by Harmful here.

Perhaps Henri can explain what he means by Harmful here.

There are mainly three kinds of harmful here:

1. Exposing detail that's not currently exposed or that would be feasible to stop exposing. (The argument that the UA can choose to return bogus values that don't really expose more detail collapses this case to the third case below.)
2. Added structure making it harder to use lesser structure to have things be multiple ways at the same time for compat. Notably, Sec-CH-UA-Full-Version doesn't allow for a real Gecko version and a fake Chrome version such that there'd be a good chance that sites that haven't considered Gecko would end up seeing the fake Chrome version. (Notably, Chrome has benefited from being able to appear as Safari, Safari has benefited from being able to appear as a Gecko-based browser, and Gecko-based browsers have benefited from being able to appear as pre-Gecko Netscape for sniffing purposes. In that sense, the lesser structure User-Agent has been a feature rather than a bug.)
3. I don't see the argument that something is already available as a reason to expose it in another way. That adds more bytes on the wire / more API surface / more complexity to the Web Platform, in which case the harm is the data waste or increased platform complexity and not the identifying bits exposed.

I'd like to see an explanation of why having policy/setting/request-budget-API modify the HTTP User-Agent value and the values returned by the existing navigator APIs wouldn't solve the entropy problem that User Agent Client Hints aims to solve. (As noted above, I view structuredness less of a problem that needs solving.)

Having a lot of different headers makes it easier to Vary more granularly (assuming a distant future where you could Vary exclusively on User Agent Client Hints and not on User-Agent at all), effectively leaking a CDN-side concern (since ISP-side caches are no longer a thing) to browsers, but the spec and README only hint at that parenthetically instead of framing it as the big motivation.

Having different levels of fiction exposed via the pre-existing header / APIs doesn't seem conceptually different form returning fictitious values in User Agent Client Hints. AFAICT, if returning fictitious values in User Agent Client Hints is already proposed at this point in time, we might as well not add new headers to requests and not add API surface and put the fiction, perhaps conditionally, in User-Agent and the existing navigator APIs that we already have.

Thanks for the reply @hsivonen.

There are mainly three kinds of harmful here:

1. Exposing detail that's not currently exposed or that would be feasible to stop exposing. (The argument that the UA can choose to return bogus values that don't really expose more detail collapses this case to the third case below.)

For Chromium-based browsers, no additional information about the browser, OS, or device is exposed. For Firefox, I think the only thing that isn't regularly exposed, or possible to infer already is Android device model (please correct me if I'm wrong!).

I'll note that it is possible to expose the device model in the Firefox for Android UA string today, but as far as I'm aware the stock browser does not ship with that enabled using the general.useragent.use_device pref (I wouldn't know if there are partner distributions or Fenix forks that do ship with that on): https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsHttpHandler.cpp#988-1004

2. Added structure making it harder to use lesser structure to have things be multiple ways at the same time for compat. Notably, Sec-CH-UA-Full-Version doesn't allow for a real Gecko version and a fake Chrome version such that there'd be a good chance that sites that haven't considered Gecko would end up seeing the fake Chrome version.

This sounds like https://github.com/WICG/ua-client-hints/issues/196, and I agree we should fix this (this was raised by Microsoft as a possible compat foot-gun). I think the right thing to do here is deprecate Sec-CH-UA-Full-Version and adding something like Sec-CH-UA-Full-Version-List that allows for GREASEing on versions. I'm hoping we can make progress on that this quarter.

3. I don't see the argument that something is already available as a reason to expose it in another way.

I see your point, but one of the things we're trying to achieve - beyond fixing passive fingerprinting via the UA string - is to create a more useful, ergonomic API for developers. UA string parsing is non-trivial and libraries are prone to errors (especially when new browsers are released - I worked for many years trying to get sites to update their site bugs caused by somehow incorrectly identifying the new Firefox OS, Firefox for iOS, Firefox Reality, etc. User-Agent strings). In my opinion its desirable to create a new, cleaner API to expose this information, even if it's duplicative.

I'd like to see an explanation of why having policy/setting/request-budget-API modify the HTTP User-Agent value and the values returned by the existing navigator APIs wouldn't solve the entropy problem that User Agent Client Hints aims to solve.

and

AFAICT, if returning fictitious values in User Agent Client Hints is already proposed at this point in time, we might as well not add new headers to requests and not add API surface and put the fiction, perhaps conditionally, in User-Agent and the existing navigator APIs that we already have.

I don't know how realistically solve for passive fingerprinting in a conditional fashion at the HTTP layer. I suppose you could use a list of trackers (something like DuckDuckGo's or Disconnect's tracker lists) and modify behavior on that existing list. But sites can just create new domains and you have no idea if they're creating or sharing server-side fingerprints - how would the list ever be updated to keep up?

(As noted above, I view structuredness less of a problem that needs solving.)

We'll have to disagree on this point. So many Opera Mobile and Firefox Mobile (Firefox OS and Firefox for Android) compat issues in the early days were caused by mis-parsing the UA string, or assuming all mobile browsers would have similar looking UA strings to the popular ones for a given region. I think providing a better, more predictable API is worth solving for, both for users and developers.

In the discussion surrounding Mozilla's position change, @hsivonen stated, regarding the browser bugs use case:

This one indeed is something that browsers can’t offer designed detection surface. Still, this generally needs the major version of the engine. Not “brand”, minor OS version, or such.

I wanted to offer a few counter-examples. My employer, Cloudinary, serves a lot of media to Safari on behalf of our customers. Media decoding in Safari is largely handled by the underlying OS libraries, and we have struggled to work around a series of bugs introduced and fixed in minor macOS versions since Apple froze macOS Safari’s User-Agent header on Macintosh; Intel Mac OS X 10_15_7.

@hsivonen:

An interesting question is how relevant this actually is with current version uptakes for browsers and sites having resources to pay attention to users who for whatever reason aren’t updating according to the rapid release schedule. At the time sites deploy a workaround, they can’t necessarily know what future browser version won’t have the need for the workaround. Can we guarantee only retrospective use? Do Web developers care enough about retrospective workarounds for evergreen browsers?

We never want to ship someone a broken image/video. Part of our core value proposition to customers is handling the complexity of UA-adaptive media encoding/optimization, for them. When we encountered an earlier Safari media bug introduced and fixed before the UA freezing, our flow was to:

• Identify which types of media were affected, starting with which OS version
• Deploy new resource generation and delivery logic, keyed on facts about the content and the requests' User-Agents, to avoid sending would-be-affected media to would-be-affected end users. Also bust a bunch of caches, so that the resources behind existing URLs get regenerated using the new logic. Affected UAs ended up getting lower-common-denominator, significantly-less-optimized resources (for instance, when JP2s with alpha started acting up: PNGs instead of JP2s)
• Find or file browser/OS issues
• Monitor those issues, and when a version that fixes the bug is released, update our logic again, accordingly. The workaround logic that looks for a range of affected versions then lives on ~forever in our codebase, even though affected requests dwindle over time.

When confronting the more recent macOS media-handling bugs, we can't identify the request's OS, so we create a loose, no-guarantees mapping between browser version (which Safari still reports) and OS version. Because broken media is a much worse experience than unoptimized media, we end up being rather conservative, sending fallback formats to (sometimes many) more people than we need to during the workaround period, and nervously guessing which browser version will almost guarantee a certain OS version, before flipping the fallback switch back off.

Because there are no guarantees, we are almost certainly sending broken media to a non-zero number of end users. Also, the extra cache busting and delivery of unoptimized media doesn't come without costs for us, our customers, and end users. We are very much looking forward to a future in which customers who are particularly impacted by these costs could choose to delegate OS version information to us, via the UA client hints, allowing us to send fallbacks to precisely the end users who need them.

@eeeps thank you for providing some real world cases! This is exactly the kind of info that is often missing form these discussions.

@miketaylr wrote:

We'll have to disagree on this point. So many Opera Mobile and Firefox Mobile (Firefox OS and Firefox for Android) compat issues in the early days were caused by mis-parsing the UA string, or assuming all mobile browsers would have similar looking UA strings to the popular ones for a given region. I think providing a better, more predictable API is worth solving for, both for users and developers.

A couple of years ago (during the Firefox OS days), Dailymotion gave me a sample of HTTP user agent strings (it was around 400,000 unique UA strings and their frequency). It included every type of requests browsers and bots alike. They gave the list for access to the API, the CSS file and a static icon. This was a wonderful trove of data.

Google has Web properties with high traffic. (be careful that a sample for a limited period of time during a day will sway the data to a specific region of the world.) Is Google doing statistical analysis on the current UA strings? Are there patterns or features in this statistical set that could be explored?

My thoughts are along,

What are the UA string structure features extracted from the mass of the data (machine learning?) instead of the desired structure of the spec?

About the cloudinary use case, this is indeed a real use case which has happened multiple times, where web developers will use the specific version number of an OS, or a browser or sometimes even a model to walk around a bug, but this has a cost too.

The optics of Cloudinary is I have a bug to fix with one browser/one OS not doing the right thing at a point in time, and I need to provide an alternate way for the users. And a well maintained Cloudinary providing a Web service to deliver to users works well. This is the poster child of a good use case.

The optics of a browser vendors is there are thousands of websites using libraries that are locally deployed. These libraries have outdated strategies, bugs, and with logics such as:

• If browser_x, don't send this format (even if the browser_x supports the format in the future)
• If browser_version > nn sends [this] else sends [that] (even when nn+1 changes the strategy)
• no fallback, aka sites catering for most popular browsers.

The Webcompat team at Mozilla is catering for this kind of issues relentlessly through the webcompat.com. It takes time, it costs money and it creates a lot of frustration for users.

I agree with @miketaylr that bad parsing has always been an issue, but better parsing will not solve bad and unmaintained logic. I understand @hsivonen about increasing the variability of surface (very similar topic than Variability in Specifications) will probably increase the woes for browsers webcompat teams, issues which are affecting even more browsers with less market shares.

And we can be sure about two things:

1. that browser implementers will lie about the values provided in Client-Hints in the future because they have to for webcompat reasons
2. HTTP User Agent Strings will not go away because of legacy web properties.

Is Google doing statistical analysis on the current UA strings? Are there patterns or features in this statistical set that could be explored?

My thoughts are along,

What are the UA string structure features extracted from the mass of the data (machine learning?) instead of the desired structure of the spec?

This seems like an interesting discussion to have elsewhere, perhaps in the UA-CH spec if you'd like to file an issue: https://github.com/WICG/ua-client-hints/issues/new, or maybe in the whatwg/compat repo: https://github.com/whatwg/compat/issues/new

The optics of a browser vendors is there are thousands of websites using libraries that are locally deployed. These libraries have outdated strategies, bugs, and with logics such as:

• If browser_x, don't send this format (even if the browser_x supports the format in the future)
• If browser_version > nn sends [this] else sends [that] (even when nn+1 changes the strategy)
• no fallback, aka sites catering for most popular browsers.

Yeah, these patterns are bad, and sometimes browsers or the platform can force developers into these patterns, e.g. when there aren't proper feature detection mechanisms for a given feature, the UA string becomes the proxy for feature support. Or if a browser doesn't expose the buggy OS version number, there's not a lot of great options: browser fingerprinting? not supporting that browser? abusing non-standard interfaces to "feature-detect" a browser? Those options don't seem great.

(I recall a number of years back we couldn't remove the non-standard window.controllers from Firefox because sites were using it to "feature-detect" Gecko:

https://bugzilla.mozilla.org/show_bug.cgi?id=1010577#c5

It seems that's still the case: https://searchfox.org/mozilla-central/rev/9c91451cc2392d942a42493fc895f5aeeddde45d/dom/base/nsGlobalWindowInner.cpp#3035-3037)

I agree with @miketaylr that bad parsing has always been an issue, but better parsing will not solve bad and unmaintained logic.

Yes, agree. It's impossible to fix these mistakes in the deployed web, short of writing a lot of site-specific workarounds (shout out to the Mozilla webcompat team for unbreaking sites for Firefox users).

And we can be sure about two things:

1. that browser implementers will lie about the values provided in Client-Hints in the future because they have to for webcompat reasons
2. HTTP User Agent Strings will not go away because of legacy web properties.

re: 1 - Yes, probably. Developers don't always have the best incentives, budget, or even knowledge to make the web work for everyone, unfortunately. But that doesn't mean we shouldn't improve the passive fingerprinting surface that HTTP UA string provides.

re: 2 - Of course. Our UA Reduction plan is designed to be pragmatic and backwards compatible. Nobody is proposing to stop sending the User-Agent header.

@miketaylr discussed again today. We want to try to tease this apart a little. One question that came up : currently browsers lie in the UA string because web sites make assumptions. What tells you that the CH approach will compel UAs to tell the truth more often or will somehow disincentivize them from lying?

One question that came up : currently browsers lie in the UA string because web sites make assumptions. What tells you that the CH approach will compel UAs to tell the truth more often or will somehow disincentivize them from lying?

Hi, I maintain WebKit's UA string quirks list (currently used only by WebKitGTK). I can pretty much promise that if WebKit were ever to implement CH -- which currently seems unlikely, I believe our position on CH is neutral? -- then we would definitely lie, likely beginning from day one, and likely in the same ways and for the same reasons that we lie in the UA header. How exactly depends on which hints are mandatory to implement in the final spec. We would probably hardcode Safari for the browser name, for example, and maybe let UAs override that at their peril (not recommended). We might choose to lie about CPU architecture or operating system, etc. Unfortunately, we can be certain that telling the truth will cause websites to break in the same ways they do currently with the UA string, so we have extremely low incentive to be truthful.

I'm not sure this is necessarily a bad thing, though. I don't see CH as a way to avoid lying. CH would be a more structured alternative to the UA string, to encourage web developers to avoid accidentally breaking things by misparsing the UA string. Reducing accidental breakage seems good for me. But there will always be websites that decide they do not want to allow access to FreeBSD users or s390x users (yes, these run web browsers!) or whatever, and that means we'll always need to lie.

Same here than what @mcatanzaro said about WebKit see https://github.com/WebKit/WebKit/commits/main/Source/WebCore/page/Quirks.cpp and https://github.com/WebKit/webkit/blob/main/Source/WebCore/platform/UserAgentQuirks.cpp

and for Mozilla, our team (Mozilla Webcompat team), we do the same things. https://github.com/mozilla-extensions/webcompat-addon/blob/main/src/data/ua_overrides.js https://github.com/mozilla-extensions/webcompat-addon/blob/main/src/data/injections.js

Ok, so it really comes down to wether having the new structured data is worth the implementation effort here. There appears to be some value for developers - despite the fact that all UAs would continue to (rightfully!) lie.

Without committing to implementation, are Mozilla and WebKit supportive of moving forward with standardization?

I'm asking as Chair of the working group who would take this work on.

I can't provide an official answer for WebKit. CC: @othermaciej

Ok, so it really comes down to wether having the new structured data is worth the implementation effort here.

Personal answer: CH is surely not any worse than what we have now. But WebKit's bar for feature inclusion is pretty high, and the benefit doesn't seem great.

What would really incentivize implementation would be to combine CH with some very aggressive plan to make it harder to detect what browser is in use, e.g. remove the Sec-CH-UA hint from the spec, then pair CH with a plan to pare down existing UA strings on all browsers, even if it temporarily breaks some websites. Think "remove the string 'Chrome' from the user agent" level of change. Then I could get excited about CH. That would be a pretty huge change though, and I doubt it's in the cards.

What you propose sounds reasonable, @mcatanzaro. We should also have a similarly high bar for standardization, as per WebKit - as without implementer backing, there is little point in doing the standardization work.

@karlcow, should we drag in a few more Mozilla folks? Maybe @tantek, as he was at present at the meeting with the Google folks. I see Mozilla's position on this remains marked as "harmful", and it echos @mcatanzaro concerns above.

The relevant parts of Mozilla's objection to this proposal:

In addition to not including this information [Sec-CH-UA], we would prefer freezing the User Agent string and only providing limited information via the proposed NavigatorUAData interface JS APIs. This would also allow us to audit the callers. At this time, freezing the User Agent string without any client hints (which is not this proposal) seems worth prototyping. We look forward to learning from other vendors who implement the "GREASE-like UA Strings" proposal and its effects on site compatibility.

@miketaylr, @yoavweiss, as the folks proposing this, are you willing to reduce the design to align more closely with what WebKit and Mozilla are requesting (maybe drop Sec-CH-UA entirely, work together to freeze the UA string, and maybe just expose the JS API)?

@miketaylr

I don't know how realistically solve for passive fingerprinting in a conditional fashion at the HTTP layer. I suppose you could use a list of trackers (something like DuckDuckGo's or Disconnect's tracker lists) and modify behavior on that existing list. But sites can just create new domains and you have no idea if they're creating or sharing server-side fingerprints - how would the list ever be updated to keep up?

Currently, conditional exposure of low vs. high entropy UA Client Hints is being proposed. Apart from the structuredness concern, what would be worse if the same mechanism controlled low vs. high entropy User-Agent value instead?

(This should not be read as an endorsement of the proposed opt-in mechanism.)

Hi everyone, lots to reply to. I will try my best.

@torgo wrote:

One question that came up : currently browsers lie in the UA string because web sites make assumptions. What tells you that the CH approach will compel UAs to tell the truth more often or will somehow disincentivize them from lying?

UA-CH wasn't design to prevent browsers from lying, and I don't think any technology can account for the many constraints (technical or otherwise) that cause developers to gate experiences against allow- or block-lists for browsers (as much as we might disagree with these decisions).

To recap, UA-CH is attempting to solve 2 problems: 1) reduce passive entropy sent over the network, and 2) break free from the legacy hot mess that is the UA string by providing an actual API (HTTP and JS). However, part of 2 tries to account for the reality that sometimes browsers need to lie to get things to work. Revisiting https://wicg.github.io/ua-client-hints/#grease is probably worthwhile.

So I think the framing is backwards a bit - with brand lists in Sec-CH-UA, we're trying to disincentivize sites from creating inflexible allow- or block-lists, or sniffing for a single ossified string. In a way, this allows for UAs to both lie and tell the truth without punishing users, e.g., you could imagine a site that is sniffing for "Chromium" in the brand list - Firefox could send:

sec-ch-ua: "Chromium";v="92", " Not A;Brand";v="99", "Firefox";v="99", "Gecko"; v="99"

That should unbreak the site, and Firefox usage doesn't disappear from the analytics of a given site, with all the expected negative consequences (which is a trade-off that needs to be considered when doing UA spoofing today). Similarly, some WebKit port could always send "Safari" in addition to "WebKitGTK" to avoid being told it wasn't supported on certain sites (or whatever the right values would be, I admit I don't know a lot about the WebKit embedder ecosystem).

Sec-CH-UA doesn't prevent sites from being inflexible - there's plenty of ways to accomplish that. But hopefully it provides useful flexibility to browsers to benefit their users.

I probably should have read @mcatanzaro's reply before writing the above, because I think we're in agreement. Particularly here:

I'm not sure this is necessarily a bad thing, though. I don't see CH as a way to avoid lying. CH would be a more structured alternative to the UA string, to encourage web developers to avoid accidentally breaking things by misparsing the UA string. Reducing accidental breakage seems good for me. But there will always be websites that decide they do not want to allow access to FreeBSD users or s390x users (yes, these run web browsers!) or whatever, and that means we'll always need to lie.

@mcatanzaro also wrote:

What would really incentivize implementation would be to combine CH with some very aggressive plan to make it harder to detect what browser is in use, e.g. remove the Sec-CH-UA hint from the spec, then pair CH with a plan to pare down existing UA strings on all browsers, even if it temporarily breaks some websites. Think "remove the string 'Chrome' from the user agent" level of change. Then I could get excited about CH. That would be a pretty huge change though, and I doubt it's in the cards.

This reminds me of a conversation I had with @yoavweiss at TPAC when I worked for Mozilla, and he proposed all browsers sending the same frozen UA string. It's certainly an interesting thought, but also a little bit scary when you consider what might break for folks to get there, especially in the long-tail of the (unmaintained) web. I also wonder if this pushes developers to get more creative in their fingerprinting techniques to work around engine bugs, or sniffing for platform features where the feature detection story isn't great. Something to think about (and a great reason to improve the feature detection story for the platform).

@marcoscaceres wrote:

I see Mozilla's position on this remains marked as "harmful"

And then quotes part of the rationale:

In addition to not including this information [Sec-CH-UA], we would prefer freezing the User Agent string and only providing limited information via the proposed NavigatorUAData interface JS APIs. This would also allow us to audit the callers. At this time, freezing the User Agent string without any client hints (which is not this proposal) seems worth prototyping. We look forward to learning from other vendors who implement the "GREASE-like UA Strings" proposal and its effects on site compatibility.

Fun fact, I helped to write that rationale back when Mozilla considered UA-CH "non-harmful", jointly with :mt, IIRC. Not sure if they meant to keep that original language, or if it was left in unintentionally (the "[w]e look forward to learning from other vendors" part in particular doesn't quite jive with "harmful"). It may be useful for someone from Mozilla to make sure it reflects their current thinking. (I'll also note that auditing origins that request client hints via Accept-CH is also possible at the HTTP level, not just the JS interface.)

@miketaylr, @yoavweiss, as the folks proposing this, are you willing to reduce the design to align more closely with what WebKit and Mozilla are requesting (maybe drop Sec-CH-UA entirely, work together to freeze the UA string, and maybe just expose the JS API)?

I'm willing to consider a lot of things, especially if there is genuine implementer interest. :) The most useful thing for folks proposing these kinds of changes would be to file issues on the repo. I think getting rid of the HTTP component of UA-CH wouldn't be an improvement, but that's a conversation that can be had in a relevant spec issue.

@hsivonen wrote:

Currently, conditional exposure of low vs. high entropy UA Client Hints is being proposed. Apart from the structuredness concern, what would be worse if the same mechanism controlled low vs. high entropy User-Agent value instead?

Are you suggesting a default low-entropy UA string (this is Chrome's current thinking for the Reduced UA string), and a client hint to opt into a high-entropy UA string? Do you think a site should get all the entropy, even if it might just need platform version, for example?

In a world where something like Privacy Budget existed, if I have a given entropy budget I would want to only request exactly what I needed to avoid going over. If my only choice is "low" vs "high" entropy UA string with more info than I need, that perhaps creates a challenge for me.

The decomposed UA client hints allow me to be more granular in my requests. For example, I can ask for device model if that's the only thing I need to verify a mobile browser is capable of handling a device-specific Android intent (feels a bit contrived, but consider the Firefox Mobile UA override for the Samsung Galaxy store).

(I might misunderstand your question or thinking, feel free to correct me).

Thanks @miketaylr. I filed a bug over in the other repo proposing this becomes two separate specs (http headers + JS API): https://github.com/WICG/ua-client-hints/issues/249

Not sure if that will make things more palatable for potential implementers, but... maybe 🤞?

I also wonder if this pushes developers to get more creative in their fingerprinting techniques to work around engine bugs, or sniffing for platform features where the feature detection story isn't great.

Authors assuming a correlation between e.g. engine-specific properties existing on window and a particular quirk existing indeed is more problematic to deal with after the quirk is gone than faking User-Agent or Sec-CH-UA.

Currently, conditional exposure of low vs. high entropy UA Client Hints is being proposed. Apart from the structuredness concern, what would be worse if the same mechanism controlled low vs. high entropy User-Agent value instead?

Are you suggesting a default low-entropy UA string (this is Chrome's current thinking for the Reduced UA string), and a client hint to opt into a high-entropy UA string? Do you think a site should get all the entropy, even if it might just need platform version, for example?

I'm asking what if the opt-in mechanism was the same (or functionally equivalent) to what you a proposing for high-entropy UA Client Hints. So to the extent what you are proposing is granular, I'm asking about a granular opt-in scenario for now.

@hsivonen Can you re-phrase, perhaps with an example? I don't fully grok your request. As I understand it, client hints is granular and allows for granular opt-in.

Can you re-phrase, perhaps with an example? I don't fully grok your request. As I understand it, client hints is granular and allows for granular opt-in.

Currently, UA Client Hints doesn't send Sec-CH-UA-Platform by default has an opt-in mechanism where the server sends e.g. Accept-CH: Sec-CH-UA-Platform and then the browser starts sending the platform in Sec-CH-UA-Platform.

The corresponding alternative that I'd like to understand the evaluation of would be:

1. By default send a User-Agent header that claims Windows even on non-Windows platforms.
2. If the server sends Accept-UA: platform (making up syntax for illustration), subsequently the browser sends a User-Agent header with the real platform (which may still be Windows). (And don't send any Sec-CH-UA-* headers.)

Currently, UA Client Hints proposes getHighEntropyValues for obtaining the platform via JS.

The corresponding alternative that I'd like to understand the evaluation of would be:

1. By default, make navigator.platform return a value corresponding to Windows regardless of actual platform.
2. If the page calls navigator.getHighEntropyValues(["platform"]) and the resulting promise has successfully resolved, thereafter make navigator.platform return a value corresponding to the actual platform (which could actually be Windows). (And adjust the value returned by navigator.userAgent accordingly as well.)

(The point here being that this kind of arrangement would leave less detritus around if the initiative fails whereas there is currently a real risk that if the initiative fails, the headers that Chrome now sends by default, Sec-CH-UA and Sec-CH-UA-Mobile, will stick around approximately forever.)

Currently, UA Client Hints doesn't send Sec-CH-UA-Platform by default has an opt-in mechanism where the server sends e.g. Accept-CH: Sec-CH-UA-Platform and then the browser starts sending the platform in Sec-CH-UA-Platform.

This used to be true. In https://github.com/WICG/ua-client-hints/pull/221, Sec-CH-UA-Platform was changed to a default, low-entropy hint (which was implemented in Chrome 93 and should ship to the stable channel later this month). The rationale can be found in the linked issue, or in this doc I wrote up (the tl;dr is compatibility concerns and platform is passively observable anyways, unless you're behind a network proxy, for example).

The corresponding alternative that I'd like to understand the evaluation of would be: By default send a User-Agent header that claims Windows even on non-Windows platforms. If the server sends Accept-UA: platform (making up syntax for illustration), subsequently the browser sends a User-Agent header with the real platform (which may still be Windows). (And don't send any Sec-CH-UA-* headers.)

This is partially the original design of UA-CH (default everything to Windows 10), but as that linked doc describes, it's not currently shippable on the web without breaking some important a11y use-cases on legacy sites (primarily keyboard shortcuts).

(Note: It's entirely possible that my conclusions are incorrect. This is where another browser engine experimenting with these ideas would be useful!)

Like @marcoscaceres suggested in https://github.com/w3ctag/design-reviews/issues/640#issuecomment-896413026, I'm open to doing the work to split the spec between HTTP and JS APIs if Mozilla would like to experiment with your proposed alternative through the JS interface. https://github.com/WICG/ua-client-hints/issues/249 would be a good place to follow up on that. If you all find that's a workable solution, I am willing to incorporate that feedback into the UA-CH spec.

These days one of the main uses of UA sniffing is to send different JS bundles, has there been consideration of a client hint that exposes the supported level of the ES spec?

https://github.com/whatwg/html/issues/4432 is a relevant discussion for a related markup-based feature, that didn't go anywhere due to lack of consensus.

There have been Google-internal initial conversations about a "browser build year" hint/markup that can partially tackle that use-case, hopefully in a less controversial way. I'm hoping those conversations can be made public soon.

At the same time, I don't think this is directly related to UA client hints, but can be discussed separately.

@plinss - @jridgewell just published a document on the front of ES support levels.

In WICG/ua-client-hints#221, Sec-CH-UA-Platform was changed to a default, low-entropy hint (which was implemented in Chrome 93 and should ship to the stable channel later this month).

That the set of sent-by-default Sec-CH-UA-* headers has grown already does not look good for the notion that UA Client Hints is opt-in.

Header compression makes the number of bytes fewer than the obvious HTTP/1.x length, but even with compression, there's non-zero overhead to sending the default UA Client Hints to every site.

FYI, we published an update of our UA Reduction plans today with projected timelines and Chrome milestones: https://blog.chromium.org/2021/09/user-agent-reduction-origin-trial-and-dates.html

A couple of additional questions:

1. Was introducing a HTTP cache feature for varying on parts of the User-Agent header considered? (For example, instead of introducing Sec-CH-UA-Mobile in order to be able to say Vary: Sec-CH-UA-Mobile, introducing a response header like Vary-User-Agent: Mobi with the semantics that the content varies based on whether the User-Agent header had the substring "Mobi".)
2. Considering that there's a proposal to add a TLS-layer CH opt-in mechanism (which in itself looks like rather excessive complexity as the first impression), has not sending any Sec-CH-UA-* headers by default (i.e. requiring TLS opt-in even for the basic stuff) been considered?

Here's a consequence of header proliferation that doesn't depend on the number of bytes but on the number of headers: https://twitter.com/ericlaw/status/1440051277564551181

@hsivonen Dug into this a little; for Apache, at least, the default limit has been 100 headers for at least 22 years. Do we have any way to see how many people may have customized this down, such that 16<n≤100 headers results in failures?

It turns out that the Rust httparse crate shows 16 in its README. This has been copied around. Mozilla has run into this previously.

It turns out that the Rust httparse crate shows 16 in its README. This has been copied around. Mozilla has run into this previously.

Ah, sad - I remember that bug...... I just filed a PR on the README to be consistent with the docs examples (which is 64): https://github.com/seanmonstar/httparse/pull/105

edit: PR was merged.

has not sending any Sec-CH-UA-* headers by default (i.e. requiring TLS opt-in even for the basic stuff) been considered?

This is already true for non-secure sites (for any client hints, not just UA-CH).

And I agree that ACCEPT_CH is fairly advanced for most use cases, which is why there's a plain HTTP Critical-CH mechanism to get similar functionality. But I think requiring ACCEPT_CH to get any UA-CH puts the entire feature set, and all its benefits, out of the reach of the vast majority of web developers.

Hi @miketaylr can you give us an update on this? It looks like Mozilla's position has shifted and https://mozilla.github.io/standards-positions/#ua-client-hints it looks like you're shipping this in Chrome 98 https://chromestatus.com/feature/5703317813460992. Have the issues that were raised in the TAG session been taken into account? If so, we'll likely close this review positively.

Hi @torgo, thanks for the ping. Yes, I believe we've taken Mozilla's feedback into account, both by shipping the Sec-CH-UA-Full-Version-List hint, as well as trying to make the spec more clear that it does not require browsers to expose more entropy than they currently do in the User-Agent string.

We think that what is proposed is a very major change to the way the Web works. On the face of it what is contemplated is "merely" a change to the use of the User-Agent HTTP header field and introduction of other HTTP fields, altering the interaction between client and server.

However the consequences are much wider than this and affect almost all the related systems that go to make the Web work. We are therefore very concerned that the proposals will “break the Web” in various ways, and therefore require very close scrutiny.

• Use of the User-Agent field in its present form is understood to be an accretion of various custom and practice over many years, it is untidy but it works. • It’s understood to be a potential privacy problem as potentially exploited by unscrupulous Web site operators. It’s said to be a cause of mis-operation. It’s said to be the cause of other problems, however evidence has not been presented as to the extent or severity of any of such effects. • It’s known to be useful to honest Web site operators in detecting fraud and other unscrupulous behaviour by fraudulent Web client software, robots etc. These anti-fraud applications will cease to operate in their present form. • It is known to be widely used by benign applications to support the effective operation of the Web in other ways - e.g. CDN has been mentioned in this context. These applications also require the User-Agent in its present form.

We think that proper standardisation scrutiny of changes to established standards and their established use is required:

• The changes proposed in the form in which they have been proposed have not been justified – for example browser vendors are free to change their User-Agent header to reduce passive profiling, no change to HTTP headers is needed. • We have not seen a measured argument of how the supposed positive effects of any such changes are to be weighed against the undoubtedly negative effects of the changes. The form in which changes have been proposed stand a good chance of “breaking the Web”. • None of the proposals entailed, either at IETF or at W3C have been proposed for standards track scrutiny. Because of the highly disruptive potential of these changes, standardisation scrutiny is required.

We note that the changes discussed, although still in draft and unstandardised form are finding their way into live implementations.

We will be grateful for TAG consideration of these points.

Many thanks

Jo Rabin CTO 51Degrees

We read that Google is proceeding with this programme as of release M101 which we understand to be scheduled for the start of Q2 2022. We are very concerned about the timeframe for this introduction and its implications on our customers and their users and the Web in general. As we describe below, we consider the timeframe to be reckless.

We operate a “device detection” service which our customers use for various purposes including enhancement of their users’ experience, fraud detection, analytics and other similar benign activities. Our customers tell us that they have not heard about or know little about the proposals. While we have adapted our product to work according to what we understand the proposals to be, our customers require time to implement any changes to their Web infrastructure and the practicality of doing so soon is very limited.

Our understanding of Google’s proposals is in any case “at a point in time”, the stability of those proposals has not yet been established (the latest version of User-Agent Client Hints is dated Feb 9) and they have not yet achieved consensus as to their nature in any standardisation forum. We were pleased to hear about the recently announced (Jan 13) User-Agent Reduction Deprecation Origin Trial that would allow maintenance of the current UA for some time, however we don’t think that our customers will have time to introduce this before Q2 given current engineering schedules and processes.

To summarise, the market is not ready for these changes. Significant parts of the Web - for example OpenRTB - which use the User-Agent header value have not been changed. The changes have the potential to cause wholesale disruption and will at least cause a considerable level of dysfunction in the established Web environment. We therefore ask Google to postpone the changes they propose, while the basis of those proposals stabilises, some form of standardisation consensus about them emerges and giving the market reasonable time to prepare for them.

We will be grateful if the TAG will please note our further views in its considerations.

Many thanks Jo Rabin CTO 51Degrees

We read that Google is proceeding with this programme as of release M101 which we understand to be scheduled for the start of Q2 2022.

That's correct. I posted a link in this thread last September pointing to the timelines associated with User-Agent Reduction.

Hi @jorabin-51d - Regarding some of the issues you raised I believe Mike and Yoav believe that these have been addressed - see the minutes of our call on 14-Jun from last year. @yoavweiss can you provide any further detail or comment here?

Hi @torgo   No, I don’t think the issues have been adequately addressed – here are some material points:  

• We are weeks away from the unilateral implementation of a significant change to the operation of the Web
• That change hasn’t been standardised and hasn’t yet had a TAG design review
• The details of the change are still in flux (Feb 9 being the latest update)
• We see no evidence that websites are ready for the change, indeed we think that their owners are mostly unaware that a change is needed   Many thanks Jo

@yoavweiss can you provide any further detail or comment here?

@yoavweiss is on PTO this week, but I’m happy to reply.

@jorabin-51d writes:

We are therefore very concerned that the proposals will “break the Web” in various ways, and therefore require very close scrutiny… …The form in which changes have been proposed stand a good chance of “breaking the Web”.

I really appreciate the concern for compatibility - it was a guiding principle when we designed UA Reduction and its phased rollout. Most of the use cases mentioned above rely on hints that are available by default (user experience improvements, analytics, etc.) and will not be reduced in the User-Agent header. A site won’t need to migrate to UA-CH if they only rely on browser name, major version, platform, or mobileness. Use cases like fraud detection would require extra hints, and migrating to UA-CH will allow those to continue to function.

That said, we haven’t received concrete evidence of site compat issues through our own testing or from users who have enabled the chrome://flags/#reduce-user-agent flag. We’ve also been running an Origin Trial since Chrome 95 (launched on October 19th of last year) but have not received reports of site breakage, either publicly or through partner channels. If @jorabin-51d does know of sites that will break, we welcome reports at https://crbug.com/new so we can work with the sites to get them fixed.

@jorabin-51d also writes:

That change hasn’t been standardised…

We hope to bring UA-CH to the W3C (and have already stated as much in this thread) once another implementer has stated an interest in implementing it.

Client Hints (which UA-CH builds upon) is standardized in the IETF.

We recently started the work to document User-Agent Reduction patterns by Firefox, Chrome, and Safari in the WHATWG. There’s more work to be done there, and it’s on my plate over the coming months.

…and hasn’t yet had a TAG design review

I’ll also note that we’re having this discussion in the middle of a TAG design review. :)

Thanks for your reply @miketaylr

I believe that this topic still requires a lot of discussion. We stand by the points we have already made about the changes proposed being reckless and that they will cause harm to the operation of the Web and associated ecosystem of services, applications etc. Rather than repeat those points, which like I say bear further discussion, I will try to keep my response as brief as I can:

A site won’t need to migrate to UA-CH if they only rely on browser name, major version, platform, or mobileness

In general that’s not the case. One of the strongest use cases is for device type, for example.

but have not received reports of site breakage, either publicly or through partner channels. If @jorabin-51d does know of sites that will break, we welcome reports at https://crbug.com/new so we can work with the sites to get them fixed.

We don’t think that gross mis-operation in the form of 500 errors etc. is the primary concern. Of more concern is user experience and optimisation. Site owners are always keen to respond quickly on first request with the best experience possible. I’m not sure I see the point of millions of sites reporting that the initial and subsequent experience for their users has become poorer, since that is a feature of the design.

We hope to bring UA-CH to the W3C (and have already stated as much in this thread) once another implementer has stated an interest in implementing it.

It is a very significant concern that there is no other implementer, since this means that the Chromium way of doing things will splinter the Web. There is a good chance that this fracture in the Web will continue indefinitely. Were another implementer to come on board it seems likely to me that the standard would move on from today’s specification with potential further disruption to website operators. We face a world of “the chromium way” and “the standard way”. We know the TAG takes a strong interest in proposals being broadly supported and of wide applicability, and we think that the evidence says this is a standard that is not reached by this proposal.

Client Hints (which UA-CH builds upon) is standardized in the IETF.

Noting that this RFC is “Experimental” and is therefore not standardised at IETF. For the convenience, here is what IETF says (RFC 2026) about Experimental:

Specifications that are not on the standards track are labeled with one of three "off-track" maturity levels: "Experimental", "Informational", or "Historic". The documents bearing these labels are not Internet Standards in any sense.

I’ll also note that we’re having this discussion in the middle of a TAG design review. :)

Noted. However the plan to proceed appears to be independent of what this TAG review will conclude.

Once again hoping that the TAG will find the content of this dialogue informative as to its findings.

Many thanks

Hi @jorabin-51d,

A site won’t need to migrate to UA-CH if they only rely on browser name, major version, platform, or mobileness

In general that’s not the case. One of the strongest use cases is for device type, for example.

Can you clarify what you mean by device type?

We don’t think that gross mis-operation in the form of 500 errors etc. is the primary concern. Of more concern is user experience and optimisation.

I agree that 500s are not the only kinds of concerning site breakage (and in my experience 400-level errors are more common than 500-level for UA changes). If you know about concrete UX or optimization bugs, or just things generally not working as expected please file them.

It is a very significant concern that there is no other implementer, since this means that the Chromium way of doing things will splinter the Web. There is a good chance that this fracture in the Web will continue indefinitely.

It's fairly normal for one engine to ship ahead of other engines, depending on the feature. That's not a new thing. Chrome is actually lagging when it comes to User-Agent Reduction efforts: Safari gets a gold medal from me 🥇 for being the leader, I think, with Firefox coming in second place 🥈 (reasonable people can disagree with the ordering there 🤷 ).

But I will note that other browsers like Firefox and Safari have not offered a way to recover the information lost to reduction. I'm not sure that's a better outcome for sites. That said, I'm hopeful they will see the value in the UA-CH API and eventually implement it, and am happy to work with them on the spec and in whatever eventual working group it ends up in to address their feedback.

Were another implementer to come on board it seems likely to me that the standard would move on from today’s specification with potential further disruption to website operators.

Are you saying another implementer is going to make things worse? Or am I missing the point?

Noting that this RFC is “Experimental” and is therefore not standardised at IETF.

OK. I think we have different definitions (and spellings!) in mind for standardization.

However the plan to proceed appears to be independent of what this TAG review will conclude.

Maybe there's some misunderstanding of what role TAG a review plays in the Blink Process. The Chromium project asks for the TAG's review and advice when it comes to technical designs and how they fit into the broader platform, not for advice on product decisions like shipping or timelines.

(An early review by the TAG seemed pretty positive on the concept in general, FWIW.)

Hi @miketaylr

I believe I have made my points and while I am always open to discussion regret that as a small company I do not have the resources to continue this to and fro in this forum.

Hi -

We are going to close this review with an outcome of "satisfied with concerns." In the main, we are happy with this proposal including the proposed use of client hints to make up for the lack if discoverability via the UA string.

The concerns we still have are about multi-stakeholder, compatibility and standardisation venue.

We are concerned about multi-implementation. We would like to see this taken up by other user agents.

We would like to the proposers to ensure that they are receptive to the concerns raised in the community regarding web content compatibility. We urge you to be transparent about publishing results of studies regarding compatibility. We also encourage the community to raise concerns in a transparent and dispassionate manner.

Although we're satisfied that the client hints spec itself is on a firm standards track footing, we're concerned about the venue for the standardisation of the ua-client-hints spec. We would like to see the ua-client-hints spec move to a proper w3c working group such as Web Applications.

Apologies for my delayed reply here... I fully agree with what @miketaylr said.