#632: User Preference Media Features Client Hints Headers

Visit on Github.

Opened May 12, 2021

Ya ya yawm TAG!

I'm requesting a TAG review of User Preference Media Features Client Hints Headers, a set of HTTP Client Hints headers around user preference media features as defined by Media Queries Level 5.

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG (in the future)
  • The group where standardization of this work is intended to be done ("unknown" if not known): unknown
  • Existing major pieces of multi-stakeholder review or discussion of this design: https://github.com/w3c/csswg-drafts/issues/4162
  • Major unresolved issues with or opposition to this design: Should Sec-CH-Prefers-Reduced-Data be defined for consistency, or yield to the existing Save-Data header?
  • This work is being funded by: Google

You should also know that...

This is a feature we (Google) are hoping to use for Google Search (google.com).

We'd prefer the TAG provide feedback as (please delete all but the desired option):

🐛 open issues in our GitHub repo for each point of feedback

Discussions

2021-05-31

Minutes

Rossen: will to ahead and post a comment here.

Dan: I don't think they need to block...

Rossen: i will post an issue on fingerprinting - easier evasion.

2021-08-30

Minutes

Rossen: it's shipped

Dan: it says early review

Rossen: shipped in chrome 93, they have pinged webkit and mozilla for input but I don't see anything more

Dan: close it?

Rossen: not much else we can add here. Everything that needed to be expressed was done. I can write a closing comment and close it. Still no position from Mozilla or feedback from webkit. Tess, do you remember what you think about client hints? Especially the media feature client hints?

Tess: there have been a lot of discussions.. I do'nt know specifically

Rossen: this one is exposing all of the CSS media features as client hints

Tess: quite unlikely there would be enthusiastic support from us

Dan: mozilla is not saying anything. Given lack of implementer support, what's the venue intended to be? Where does this go? Right now it's in WICG.. Where does it go next?

Rossen: not much going on in repo since they moved it to WICG

Dan: let's close it. But what result.

Rossen: no multi-stakeholder. despite the fact that it's shipped it's supposed to be an early review. I'm glad they reviewed this with their security team. Satisfied with concerns?

Dan: yeah

Peter: the whole client hints thing seems contentious and up in the air

Dan: we have previously reviwed client hints positively

Rossen: way back when, then we reopened it

Peter: my fundamental problem with this is it's built on top of a feature which doens't have cross vendor buy in and still contentious and isn't really on a rec track anywhere. This is a google feature, which is fine, but not a standard.

Rossen: so close as unsatisfied?

Dan: ambivalent because we don't see it's harmful however it's got these signifcant issues with stakeholder support

Peter: it's not on track to be a w3c standard. I think we need a different status, not sure ambivalent captures it. More like.. out of scope. I don't want to give the TAG's blessing that this is a standard.

Rossen: good to capture that in the closing comment

Peter: my concern is not to signal TAG approval for something that isn't really participating in REC track standards

2021-09-20

Minutes

Dan: Rossen left a comment last week, they've got back to you

Peter: not sure they got the point of rossen's comment.

Rossen: they're saying we need client code to change so that you're pulling all the time?

Peter: "..always expected to require new resources" - I don't think that's true

Rossen: it's not. When I go to print I don't necessarily have to go and redownload everything

Peter: possible to do that if you want to shave a few bytes of some stylesheets or you have significantly different media, but not a requirement.

Rossen: either we're talking past each other.. but I'm surprised. Yoav's part is more about where the work happens. I'm surprised by Thomas' response because I thought i was pretty clear when I explained the fact that styling is clientside operation today. I can see having a document that relies on clinet hints only for downloading everything

Dan: and you get a static html?

Rossen: you're in light mode for example, you make your initial request, client hints say you're in light mode, get a bunch of resources for light mode, done. The next thing is they go into a darker room and their device automatically switches into dark mode. In that scenario where this document was written using client hints only because the platform allows me to just rely on client hints only, so now I'm going to send a UA client hint that says dark, which means the server needs to recognise that you went from a light to a dark mode, that's a change, I need to send a whole bunch of new content, stylesheets, images, etc. Which means the state is now being transferred over to the server side

Dan: is that really what they want? That can't be what they're designing for

Rossen: this is how the current feature will work. I don't believe they want this.. I'd be surprised. However you're giving me an API I'm going to use that API and nothing else. So now I'm changing the way i communicate about media features to be directly on client hints only. This gives me performance.. The point is when you change the API which is currently available on the client only the client does all of the different media or feature detection, change detection. Through style recalc it recognises what needs to be changed and fetched. This is why users are adding all existing logic for dealing with media changes inside the document - light stylesheet, dark stylesheet, etc. They're setting up all of that on the clientside so when the client starts making decisions locally they can fetch the things that the authors expect them to fetch for that particular media environment. Put that on the server side as a set of hints, now as an author I don't have an incentive to put all of that stuff on the client. I can say the server will take care of it somehow. I'm hinting that i"m in dark mode or want reduced motion so the server needs to give me that stuff. The first time I navigate there the server will take the ints and give me the right stuff. But as the hints change the server needs to recognise this and say you just changed from light ot dark so I need to give you a new stylesheet. That change detection needs to happen on the server, and that is a very different model than the way styling happens today

Peter: I think they're coming from the author would still put the logic clientside and this would inline or optimise.. you can design all the logic clientside and do a small optimisation using this hint, or you can totally screw it up and do all of the logic server side

Rossen: you're promising this api is going to be so much faster and better, I'm going to jsut use this API, and it's the only thing I'm going to use. Now I screwed up everything because media detection needs to happen on the server side.

Peter: I accept it's an opportunity for small optimisation. You already need the round trip for the client hint. By the time you've made the round trip you could have just pushed all the logic... even if it's used properly I'm not sure it's buying all that much

Rossen: let's say for very heavy traffic websites it buys a lot.. Minor saving on initial load, however it comes at the cost of changing style, media logic to be pushed onto the server side. In the worst case where I only depend on client hints. We just changed everything.

Peter: one thing Tom mentioned is the server could inline some stylesheets.. not sure that buys a whole lot of optimisations in an h2 world

Rossen: agree. Server inlining styles great.. but you're still pushing logic onto the server side

Dan: what's our suggestion? Cut the feature?

Rossen: that would be my proposal

Dan: if it really overcomplicates things for developers by creating these two different paths

Rossen: duplicating capabilities. We don't do that anywhere else. [will leave comment]

2021-09-Gethen

Minutes

@plinss, @cynthia and myself did another review of this during our Gethen vf2f. First of all, thank you for following up on with your security team and getting an LGTM from them. I personally won't push on this further given it's already captured.

Another issue with this proposal is the affect it has to existing styling patterns. All conditional styling today takes place on the client as a result of style recalc in combination of various environment changes - color scheme, reduced motion, transparency etc. Exposing this through client hints will promote such conditional handling on the server side which will arguably negate some of the perf wins described and also require that state is kept on the server side. Ex. UA navigates to a.com in light mode exposed in the client hints. The server provides a light version of style.css and appropriate other content. The mode changes to dark in the same session - the server needs to recognize the change and this implies the client needs to reload possobly all the resources.

Finally, I don't believe CSSWG is the approriate venue for this work. It is the right place to define media capabilities, not client hints.
2021-10-25

Minutes

Rossen: already in chrome, already shipping in stable. This feature allows you to pipe media features through client hints. Through CH you can discover about the client via the headers. You can save bytes on the wire. Bigger players like this - this is cost reduction. Media features - well understood, used and deployed capability. Authors can figure out things like dark mode, reduced motion, etc... Client hints promoters say they can reduce CSS sending down the wire by using client hints. A dark mode CH for example. Having bytes. Sounds great. And along the way we've shifted a client-side capality to the server side. Round-tripping model of CSS through the server. The server has to be able to track and respond to user state.

Dan: happens to me all the time with twitter in the morning as my device changes from dark to light mode. This is a real thing.

Rossen: Right. Some discussions on this... They say "this feature is only for larger web sites like google and only for power user". Our position therefore is to close this as unsatisfied and move on. I consider this to be an anti-pattern. Regardless of how it's massaged right now - regardless of small benefit to performance it doesn't justify changing the model to have to round-trip to the server. It gets messay very quickly. I'd like to close this as unsatisfied on the basis that changing the media feature handling and style sheet loading from the client side to the server side is not good. Should be kept on the client side. Also: narrowing the use to big players is also not great.

Peter: this can be used correctly and it will save bytes - only large companies that are in complete control can use this correctly.

Rossen: the bytes saved are really tiny.

Peter: for a site as big as google.com saving 5 bytes off the wire will be signifigant. If Google want this as a proprietay feature that they've already shipped, fine. But we don't think it should be exposed to the web.

Hadley: We have an etical web principle on this.

Yves: it only matters for big companies.

Dan: fine with me.

Rossen: lea anything you heard that doesn't make sense?

Lea: No.

Hadley: finds relevant ethical principle: "We will also build Web technologies for individual developers as well for developers at large companies and organizations. The web should enable do-it-yourself developers." (It's under 'The web must enhance individuals' control and power')

Rossen: ok I will close with a closing comment.

consensus recorded

2021-10-25

Minutes

Discussed, propose closing as dissatisfied. While useful for google.com, this feature is a foot-gun for the long-tail. We don't see why this shouldn't remain a proprietary feature.

2021-11-15

Minutes

Dan: We will discuss next week.

2021-11-22

Minutes

Thomas: this feature has rolled out in chrome, we've been using it at google and it's been working fine. This is chrome status, we start seeing first page loads that make use of this. Going up. The top sites using this are whatismybrowser, and we have been using it in google internally. You might be able to see it if you go to google right now depending.. [live demo]. The client sends information to server so the server can inline the right CSS. What does this mean? Google in this barebones experience is simple, but when you search you see more things come into play. You can see there's a map component respecting dark mode. It's not just the CSS of the google page, but also all the widgets and images. In the end, the saved bytes adds up significantly. Which is why we proposed this feature in the first place, for high perf sites with lots of traffic. Francois built a boiled down version of this that shows how it can deal with dynamic traffic. Core idea is server sends initial scheme, I can mid way through manually change my colour scheme and the site dynamically updates. On demand, like any other dynamic request mechanism. By dynamically switching out the media on the link based on that, so we know if it's dark it will be light and vice versa. We can see the page dynamically pulls in the featureset it needs.

Dan: What you demo'd tries to address the issue that was raised in the review. The question I have is if you need to add a whole bunch of js into the page in order to get the end result that would have been the case anyway if you had simply used the media query in order to make this work in a way that addresses the issue of what if your computer changes from light to dark. Isn't that cancelling out the performance gain? The tension here seems to be between something that developers expect to happen on the client, purely within css, and the idea that in some cases we might want to optimise delivery of bytes, and that's important, and imporant from a sustainability perspective, but if in the end in order to make it work in a way that respects the possibility of somebody's media features changing dynamically you have to add javascript into the page, it feels like we're back where we started

Thomas: you don't have to. It has no js at all apart from this thing that makes sure we are on https. It's happening dynamically on the server. THe client has no js dependency here. The server dynamically changes the style media prefers colours cheme so when the media conditions change it will request hte missing css that is needed in the new situation. this is boiled down tot heminimum demo, it's the same logic that we have in google.com. The core logic responsible for fetching the css is all happening on the server... the html generation on the serverside based on the value we get from the request, then pass it to this html generation function.. it's just html

Dan: what happens on the client when the condition changes? The client has to do something new, it has to send a request to the server now saying my condition has changed, what triggers that?

Thomas: just done by the browser. If the media attribute changes, same if you think media print, not loaded initially, if the media condition changes - you activate print styles, or maximum width - the same mechanism here for perfers colour scheme

Dan: thanks, I get it

Thomas: what we tend to see in practice is los of site offer a dark mode toggle so it overrides the scheme for the system for the site. Can register an event listener in js that can inform your toggle that the users preferences have changed.

Peter: if you're building this without this clienthint what you'd do is link two different stylesheets with the media, if you're in light mode fetch this stylesheet and dark mode fetch this... what this feature buys is it gives this server the opportunity to know which one the client would have requested and inline that one. Shaves some bytes off the wire.

Thomas: correct.

Peter: the concern Rossen and I have is effectively, historically this was a decision made entirey clientside. You're putting a server into that flow. We accept that if you do it just right you can get a slight optimisation by inlining stylesheet, but there's a whole lot of opportunities to get it wrong. It may look right at first, but what happens if it changes dynamically, people use script to do it, the browser now lost its opportunity to prefetch that sytlsheet, there are all sorts of ways to make this worse by trying to use this feature and not getting it quite right

Thomas: [demos blog] .. fetched with lowest priority because media doesn't match. Expensiveness is I have to make the initial request... then if it needs one or the other with higher or lower priority. If this automatically changes we have the place here that the media attribute would all of a sudden match so the client fetches it. Only works if we have informatin here. If no information, we fall back to a regular flow of making the initial request and letting the client decide what it needs. This is an optimisation that will work if the client hint is being sent. Everything else is regular flow if the media attribute changes.

Rossen: What happens when you have mixed modes? You have stylesheets and documents that are cming from documents that do support or they were sent from ... how does the mixed mode work in this case? where you have half of your documents essentially sending the client hint and half don't because they were not made ready? That becomes hell for the serverside to coordinate and reconcile doesn't it?

Thomas: client hints would be sent by the browser. If the browser supports it and server requests it then itw ould respond. It's not possible that one document would send and another wouldn't becasue it's happening at the browser level.

Rossen: my point was that if you have half of your documents on the serverside not recognising the client hints because they weren't updated to, and the others do, you have a component that is old and it doesn't recognise clienthints and it's doing its styling based on that, that component is in a document that does actually on the serverside recognise clienthints. What does that lookl ike?

Thomas: if your server doesn't support this it will not request a clienthint so the browser will not send it. The server can of course decide to ignore it if the browser sends something it doesn't like. I'm showing a server that creates html dynamically based on the fact a client hint is sent. Everything else if you don't request this you don't get it. It's not a fallback, it's a progressive enhancement. You use the regular flow of showing and deciding by whatever means, it can be a manual toggle, can be perfers colour scheme media attribute of link in css. If you do css in js and clientside handle this you can do this. It's a complete optimisation for sites that want to use this feature. If there's nothing that doesn't want to make use of this nothing happens.

Peter: if this is done right - best case scenario - it saves a round trip. But in order to use this in the first place doens't the client hint cause this rt anyway? So what are we saving?

Thomas: Critical client hint - an additional opt-in mechanism. For the follow up requests sent - but for the initial request you have to say it's a critical client hint in order to see it. Our measurements show this still outperforms inlining both style sheets ... critical thing is - people having this set to automatic. Even if you have set a cookie you can't be sure it's using dark mode still.

Rossen: Way I see it - CSS today is designed to be client side tech. I can write a style sheet and ship it to any client and have it apply .. the feature that you're suggesting breaks that and requires me to write server side code ....

Thomas: [Disagreeing] This doesn't break anything because if the server doesn't request it then the client won't send it...

Rossen: If you want to create this experience for large sites then so be it... just don't see why this should be standadized...

Lea: also having trouble understanding the use cases here... there's also the media attribute on link elements... if a user changes their preference then things will start bring broken - CSS will chnage reactively and things will break... you could end up with an inaccessible intermediate state where some images assume dark mode and css assumes light mode...

Thomas: high perf sites inline everything. If you inline dark and light you will end up with a lot of css that is not used. You have to load this either way even if you don't use it. If you want to continue like today you can do that. With this optimisation the browser can also just say.. at this stage inlines critical css.. browser has same optimisations it has done before. We're in dark mode we inline dark css... light is fetched with lowest priority. Just fetches the nonmatching one with lowest priority. Were in light mode now but it has already fetcehd lowest priority lazily because it might need it. If we have a picture with a source depnendant on the mdia the browser will apply the same optimisations. Browser will run some heuristics and prefetch

Peter: we accept this can be done right in such a way that you're inlining some data, shaving some bytes, and accept that for very large sites that can get this right there are some savings. The volume of the savings add up. Our concern is that there are a lot of opportunitites to get this wrong. For people without the resources of large site we're going to see a lot of sites trying to do it half way / part way and get it wrong, and create a worst user experience and get it wrong because they're not testing walking with a mobile phone from a light room to a dark room while offline. This may propagate out to a lot of sites and cause breakage for not a lot of benefit for smaller sites because they're not at the same scale. we're wondering if it makes sense to add it to the platform. Someone does it on puprpose but doesn't quite get it right because they're not complete experts and picking it up from blog posts.

Thomas: this can happen with everything...

Peter: i understand it happens everywhere, they may break their site and not realise

Lea: a difference between things that break immediately and thing sthat break if certain conditions are true

Peter: certain conditions for certain users. We see the advantages but we're worried it's a footgun. For general user do risks outweigh advantages, or vice versa? That's our main cconern.

Thomas: no for the general user, for the general site...

Peter: does it make sense to add this to the entire platform in perpetuity, or just leaving this as a non standard feature that google has that they know how to opt into. We're not saying don't do it, we're saying should it be part of the platform?

Thomas: we're including high performance sites.. Why is something used for a high perf site not standardised where something useful in general would be?

[Yves in chat: well, having two ways of doing the same thing is always an issue, especially when one is just an optimisation for just a few selected high profile sites Also starting the trend to put all css media queries in CH would make very chatty requests]

Peter: not everything in a browser has to be standardised. If this becomes a standard I would highly recommend it includes verbiage that says it's for these conditions and not the general case, don't use this for your blog.

Thomas: we make clear this is for high perf traffic sites. Applies to all google.com users on firefox, on safari, if it was standardised. Not just chrome, browser in general, these benefits could add up universally. There are a couple of pages where this would apply - tiktok, facebook - there is a set of pages where this could benefit. If they wanted to and if they knew what they were doing they can use this. Not just in chrome ideally. We were hoping to convince others to move forward with this.

Peter: it's not that it can't be standardised, but our opinion. One thing that may help is to see some real data.. overhead to the request by adding extra headers? vs what you're gaining by inline a single stylesheet. nto comparing by inlining both stylesheets. If I was just linking both of them and using h2 and the roundtrip is faster.. Getting similar effects with alternatives to this feature

Thomas: I can ask. The blocking behaviour would only happen for the very initial request. The server is requesting these clienthints so it would respect them from then on

Peter: there is a block, it's unfair to ignore that

THomas: only until the client has learn this server respects this client hint

Peter: just make sure we're comparing apples to apples, and big picture. If we look at one part of the load we can say this part got 10%, 15% over here... data is important to look atthe broad picture

Thomas: I'll see what I can get. We did look at data or we wouldn't have pushed this forward.

Rossen: I meant to close this two weeks ago but I was away. I do expect to be able to write the closing paragraph and close it as far as the TAG review is concerned.

Peter: suggest we take time to revisit and wait for data. [defer to f2f]

Thomas: encourage everyone to look at the demo / demo and https://blog.tomayac.com/ (dynamic media attribute case)

2021-12-13

Minutes

closing

2021-12-Madripoor

Minutes

Dan: I think they were going to come back to us with some data...

Rossen: i don't question the performance improvement... we're not arguing about that... they were however using this as the primary motivation .. if they bring it data great - it will only validate that statement. And then there's the rest of the web and patterns that have existed for 20 some years...

Dan: In defense of the requtors - they have talked about other stakeholder feedback... Facebook, tiktok, etc...

Rossen: Of course, all of these comapnies will all say yes to any small feature that will save them bandwidth. Again, this is a point I agree with.

Dan: maybe we shoiuld provide them advice about guard rails to put around this rather than telling them to go away and do it outside of standards...

Rossen: not targeting the use case of saving some momey on a big property - it's about the model of media feature state tracking... and handling. Which by definition is a device capability that is changed by the enviroment or user action on that device. Devices go back and forth (e.g. dark mode)... So that state is handled from the user pov - ...

Peter: on initial load the server should inline the appropriate asset... if you do it all correctly - the logic stays on the client but the server still has responsibility to ship both assets. Our concern is that developers will do this wrong ... and not ship both assets... If you don't get it just right you've broken it...

Rossen: the set of features here... apart from Light Mode / Dark mode is large...

Dan: http3/2 push... Yoav said ... yes some browsers never shiped push... the network stack doesn't know about it... he thinks the practice of doing server push on both... not good. I'd like to see real world data... I think it fixes the round trip issue but not the delay issue...

Rossen: the fact that you're not .. that this is the case.. establishing tls handshake etc...not necessarilly a good thing. If i'm penetrating a network and want to go look for specific veunerable devices based on their state... this gives me more feedback that's much harder to detect and protect against.

Peter: kind of true for all client hints... TLS handshake you can avoic by using H2. I accept the fact that they will do it anyway... Are we better off not having a w3c standard? I don't know. If it's a standard it should have big red letters over it saying "don't use this unless you're google/facebook"

Dan: could we ask them to put mitigations in place...

Peter: [ekeptical that such mitigations could work]

Rossen: besides light mode / dark mode - you have contrast, color management, accessibility, viewport etc. etc. - a permuation of many media features. Assume we have a device in "i'm in dark mode; i'm also in high contrast mode". Now it's less dark so not in high contrast any more. The UA will send the UACH that it's not in high contrast. Back to their motivation - in order to preserve bandwidth CSS is inlined as much as possible to the user preferences that they detec - btw, today this is done using a media query on the client. They have to cascade and inline the resulted rules on the server side - the combination between dark mode and low contrast in the current example. So now all the rules that have to cascade and inline are calculated on the server side. To Peter's point - add 2 or 3 more features now you're hangling cascade on the server side. This capability can be used this way, this is my assertion.

Dan: (even if not intended)

Peter: one thing in defence - you can put certain style sheets at the head of the queue so your initial paint can be faster. If you use server push just right you can make sure that those stylesheets are available pretty much at the same time. I would like to see data that this is better than server push. I think there are situations when server push might be better. Not sure this is the best solution to the problem.

Rossen: there are many of these... I don't agree with allowing all the media features... viewport segments, color management, pointer... etc...

Peter: we did ask them for data for this vs server push. If we close it off we can say so - but please continue to explore other options to get the same benefits such as server push.

Dan: We also would like to see some thinking about mitigations against misuse?

Rossen: from a pure technical PoV... They started with a great premise... can we deprecate user agent strings... let's start working on client hints... because UA strings allow you to know aboout the hardware capabilities... Ok. now they are using this for saving bytes by looking at user state based on these capabilities. From the runtime what else can we start extracting. Everything else that media feature captutes are triggered by env variables or user intent or action... This has nothing to do with the fact that this device e.g. is running windows vs iOS for example. So now we want to know what the user is currently doing so you can give them something quicker...

Peter: there's a pretty strong line here you can draw - client hints should be about the capabilities of the client, not the state of the client. A useful client could be "does this device support auto detection of light or dark mode" - but not "I'm in dark mode now."