#643: HTMLMediaElement controlsList

Visit on Github.

Opened Jun 3, 2021

Ya ya yawm TAG!

I'm requesting a TAG review of "HTMLMediaElement controlsList" to offer a way to control the native controls elements/buttons that are being shown by the user agent in order to be able to remove some features that do not make sense or are not part of the expected user experience or only allowlist a limited amount of features.

Further details:

You should also know that...

  • Chrome has shipped "HTMLMediaElement controlsList" in 2017.

We'd prefer the TAG provide feedback as:

🐛 open issues in our GitHub repo for each point of feedback

Discussions

2021-06-28

Minutes

Lea: this is about a controlsList attribute that provides a blocklist of controls not to display in media elements. It was proposed by Chrome a few years back, strong opposition from webkit, Chrome implemented anyway. Their motivating use case was that they had added a download button to serve users that needed to dwonload content, particularly in countries with slow internet connections where people cannot stream. A lot of content authors wanted to remove the download button and implemented controls from scratch which has accessibility problems. So the pushback from webkit was we don't offer a download button and we'd be forced to implement this nodownload keyword and we shouldn't have to implement stuff for features we don't support. Was supposed to be a DOMTokenList. If webkit did not implement nodownload authors could do does controlList support no download and if it doesn't they could just not display. But it's now in shipped Chrome. Also a side debate about if this is about removing controls from the toolbar or about disabling capabilities altogher. Proposed feature is to remove the control but it's still in the context menu. Authors might still want to disable the context menu which is not idea. So may point towards disabling capabilities instead of this controls list attribute. Also debate about why a separate attribute, why not turn controls list into this blocklist in the same way the download attribute works, either as a boolean or have a value, since this list only makes sense if controls are enabled. Arguement against is you need to be able to toggle controls on and off individually. Setting controls to false would throw away your controls list. Proposed instead that chrome could implement a vendor specific attribute, x-no-download, but there was pushback because we don't want more vendor prefixed attributes. A lot of suggestions about an idividual nodownload attr, standardise, but then a cornucopia of no-whatever attributes. Chrome said at the time they are implementing more features authors might want to disable in the controls. if we can't do that it prevents improving the default controls over just pointing authors to implement custom controls.

... personally the chrome arguements make more sense to me. Worse for the web to push authors to implement custom controls, avoid that at all costs. I would not want them to have to implement a vendor prefixed attirbute. Then if gecko implements a download button next? then we have a different one? Having a nodownload attribute doesn't make sense because we end up with a bunch of no-whatever attributes instead of a single feature that authors learn once, and a clear path for disabling any new features.

Rossen: do we know where webkit stands at this point? This conversation is a while ago

Lea: multiple conversations, as far as I know they're still opposed

Peter: who are we helping by disabling the ability to download video? a lot of sites go to greath lengths to do things to make videos non downloadable, but it's always to the disservice of the users. They should serve user needs, and let people download a video if that helps their experience. Arguments about not wanting to encourage people to make custom controls, and the UA always has a clean way of downloading video..

Lea: with the feature they're proposing it's only the control on the toolbar, the download capability is still there

Peter: seems even more useless, just making it harder for a user to do something they can still do

Lea: implemented for 3 years, would be interesting to see how it's used in the wild

Rossen: that should be queryable pretty easily..

Lea: don't want to disable the capability altogether, but content authors complained they were making that visible

Rossen: 4.8%, that's quite a bit

Lea: it is

Rossen: if all of it is youtube... no, quite a lot of random websites. Yeah, it's used a lot.

Lea: a bunch of libraries for adding custom controls. Not such a huge deterrent for authors. there was the argument that if we don't add this attribute authors will just allow the download button to exist over the burden of implementing custom controls, but given they can easily drop in a library, it's not such a hurdle to them. No library can match the localisiation and accessibility of the browser.

Peter: agree, but concerned that overriding user controls over media is a dark pattern. Just because lots of people do it doesn't mean it's good. I don't know if baking a dark platform into the pattern is a good idea.

Rossen: you should add this to the design review and see what the authors come back with

Peter: just questioning higher level.. should we do this?

Lea: I agree with that. I'm torn. There's a tension. If you say users should be able to download content then you push authors to implement an even worse user experience. It's like saying no you can't customise scrollbars because the scrollbars might be hard to use, then authors end up implementing even worse scrollbars. We see this over and over.

Peter: agree there is a tension. Lots of websites do things that are bad for UX, and should we be helping them do that?

Lea: Meta point - we were asked for a tag review by chrome becuase of this dispute with apple. Is a TAG review the right place for this?

Rossen: not a bad place

Lea: tess last week said it shouldn't work that way

Peter: we can be asked to settle disputes. If that's what this is it's fair to ask our opinion. They didn't use that template. We dont' have the authority to say what they must or must not implement. Want to know higher level user needs, what benefits?

2021-07-12

Minutes

Punted to next week prior to meeting

2021-07-19

Minutes

Lea: peter asked them a question but we didn't get a response.. FWIW my opinion is - custom controls almost always on wrong side of history.. customization - as browsers add more native controls it's reasonable to not want to have some things customizable - doesn't necesarilly mean that it's user-hostile.

Rossen: i can see a lot of cases - scenarios.. taking surveys - coursework, you don't want people to skip ahead .. i can see these as valid use cases... if they didn't list those use cases that's a miss on their behalf.. not advocating for this feature or against it - can see this being applied and used on edu...

Lea: the students use case - they can circumvent by running code in the console...

Rossen: yes but.... that's only for people who can use the console...

Lea: it only takes one student figureing it out...

Dan: +1

Lea: not the major use case - with the current design that's not possible. Right now it only enables disabling specific controls - but it could be extended.... Bigger philosophical issiue at play - 2 different philosophies - this behaviour could be user hostile but if we don't allow it we could end up with much more hostile results.

Dan: what's the user hostility and what's the more hostile?

Peter: user hostility - diabling the ability to cast the video to an external device - ally issue here, I may not be able to see detail in the video on a mobile device. why can't i do that just because the author didn't like the look of those controls?

Tess: that's an argument for 2 things - allow people to implemet casting in their controls also.. also the youtube use case, no download

Peter: that's also user hostile

tess: weird that chrome has a no download feature but doesn't use it to remove the rightclick menu item

Lea: apparently feedback was authors didn't like how prominent the download button was but didn't mind it was present in the context menu

Tess: sounds like UI feedback, doesn't sound like a reaseon to add a feature to make controls mor euser hostile

Peter: I get the look and feel thing, that the app designer wants, but not to disable functionality. For that we need better tools to change look and feel. Could be abused to hide controls too, but..

Tess: separately there's an issue of how stylable should the default controls be? I think that should be a separate question but they're inextricably linked. Sometimes the only reason people resort to make thier own controls is theming. They'd be happy if they had lmited stylability of detaults - changing colours. Seems like adding ability to do some limited customisability is probably a good thing. but how do you do that without display: none or visibility: hidden?

Lea: I don't think it's always user hostile to hide controls. Add browsers add mor econtrols to native interface it's reasonable to limit how many are shown all at once

Tess: up to browse,r not the site

Lea: how does browser decide what's relevant?

Tess: look at things like size of display. If it's really small only show pause/play, etc.. user agent has the relevant context to choose what to display and not. in this context the esite is the antagonist relative to the user need

Lea: website is part of the context.. UA has no access to that

Tess: Sure. Cases where format negotiation.. conneg.. you have a video element with a bunch of source elements that are different formats and the UA might have a more granular understanding of what formats are supported than the site can. UA might know that it's currently on battery power and knows that software decoders are battery intensive so it could intentionally not choose formats it knows it support sbecuase it wants the one that gets the hardware decoder to preserve battery life over whatever the site would prefer gets loaded. The ua has a lot of context that the webpage does not. Webpage knows things the browser does not - eg. which video it would prefer to load - sometimes the ua does things the site doesn't want and that's okay

Lea: allowing authors to remove certain controls does not remove the ability of the ua to customise the default interface. ua could still change how default controls are displayed

Tess: if this controls list thing is spsecified as a hint that the ua can freely ignore i think even in that case we run the risk of sites that use the feature blocking browsers that dont respect the hint

lea: what if it's undetectable whether the browser detects the hint?

Tess: Ua sniffing. If some browser, say safari, decides to never respect the no download value of the controlslist, what you get is sites blocking safari for video playback - :(

Dan: what are the further harms? to not having this feature?

Lea: it pushes authors towards having custom controls which is much worse than any of the alternatives. Pushes browsers to not add a download button or other user friendly capabilities if authors don't want it and have no way to remove it

Tess: agree, there is a tension. I think there are plenty of user hostile things possible for websites to do. Ideally the friction of the platform shoudl be user friendly things are easy and user hostile things are hard. If we can't make it impossible we can at least make it hard. It is harder to make custom controls

Lea: only fractionally harder - including a library

Tess: library is more likely to have implemented custom controls correctly in terms of accessibility. Less user harm than without a library. That fits my gradient of difficulty.

Peter: I'd like to know this was part of their discussion. I'd like to hear what was talked about. I think there's a bigger picture here.

Tess: it's been discussed off and on for years. a pr on html years ago about this. I think a lot of this did get discussed.. and afaict both geckon and webkit are opposed to the feature. The other concern here is these people get go ahead from the tag they're going to do it anyway and have another single engine feature with dubious value.. not good for the web in the long run

Dan: the multi-stakeholder thing is an issue.

Tess: we could loop in paul (or jean yves?) from mozilla or (??) - the two most vocally opposed years ago

Dan: other people in other orgs with reelvant feedback? EFF? Privacy focussed browsers, duckduckgo? More of a user empowerment issue than a privacy issue. But since it's also related to media, useful to get feedback about whether this empowers or disempowers users

Tess: i think that's a good idea

Dan: I'll send an email. And feedback from stakeholders of the ecosystem who aren't browsers? [leaves feedback]

2021-07-26

Minutes

Lea: we need more input. Only one comment...

Tess: we @-mentioned many people on the thread and we'd like more comments first...

Dan: punted to second week in august

2021-08-09

Minutes

Tess: needs Lea

Peter: tentatively for next week

2021-08-16

Minutes

Lea: both Tess and I pinged several people but none of them have commented. Punt it more? We've had input from various people, but none of the people we actually invited

Peter: push it another week or two? Ping?

Lea: I don't know how to get these people, we have pinged them. Some other channels? We could punt and see what happens

2021-09-20

Minutes

Lea: open for a while.. push back mainly from Apple.. said we'd bring people to a special session or something. it's implemented in Chrome, nobody else seems to be interested in implementing it.

Dan: multi stakeholder red flag. Could close it satisfied with concerns, feature looks fine but nobody is implementing it?

Lea: there is a big philosphical debate around the feature.. authors should use the default control as much as possible, so if removing a button makes people use default controls so be it. Other side says if people want to do something user hostile usch as removing a download button we should make it difficult and they should use custom controls. Apple is worried that if ithis goes through people will exclude safari based on the fact it doesn't implementing the download button at all. Google wants to provide useful features and it doesn't scale to have everything visible by default. you can still right click and download the video but it's less discoverable.

Dan: If there's a debate and we don't feel like we sholud be in the middle we should close with ambivalent?

Lea: there is a position for the TAG to discuss it and decide on the larger issue here, but it's a long discussion. There is a larger issue.

Dan: plenary?

2021-12-13

Minutes

Tess: when I last looked at this Lea and I needed to have some time to work on it.

will try to schedule a special session

Peter: I'm not a fan but not sure there's a good answer either way.

2022-02-14

Minutes

Lea asked to defer

2022-02-21

Minutes

Punted per Lea's request

2022-02-28

Minutes

Punted to F2F for Lea and Tess

2022-04-04

Minutes

bumped from A so we can get Lea and Tess

2022-04-04

Minutes

Rossen: what is the disagreement? Last think I see is 6 months ago...

Rossen: my proposal is to express the disagreement in the issue ... PR seems to be landed

Peter: but still open..

Rossen: I see webkit's opposition to the feature.

Peter: two ends are allowing authors to hide controls is bad, removes platform specific controls, and is user hostile. The other half is that if we don't give them a way of filtering specific controls they turn off all controls and building their own.

Rossen: do we know if Tess or Lea are going to be around for breakout B? Let's push to B, and in the meantime try to follow the breadcrumbs and see where the discussion is.

2022-04-11

Minutes

Tess: where we left off.. we both felt fairly strongly about this, and Peter was in between us.

Peter: agree strongly with both sides

Lea: might be good to recap.

Tess: proposal from google to add a controls list attribute to html media element which will control which of the native controls show up are are hidden. There were one off things for specific controls and they wanted to generalise it. [recaps history] There are arguments that aren't about ads. One argument is if you don't give them a way to do this they'll not use native controls at all, but will use custom controls that are more bloated and less accessible, buggy. Good arguments for encouraging websites to go down the virtuous path of using the built in contorls which we believe are already accessible etc. You can see the argument that if we don't give them this feature they're going to do it anyway in possibly a worse way. There's a harm reduction argument for it. Argument against that is paving the cowpaths of user-hostile behaviour. This has been argued about for years. The tiebreaker between the two sides is the question of multi-vendor-ness. We don't have multiple vendors on board with this attribute. That's a blocker for it ever landing in the html spec. Unfortunate to give a thumbs up to something that isn't going anywhere.

... I have another idea. We could agree to disagree. There are really good arguemnts both ways. I sympathise a lot with Peter's position directly in the middle. Maybe the right thing at the TAG is for us to say we understand it's complex and there are good pros and cons. We haven't been able to come to consensus ourselves. Our understanding of pros and cons so you're informeda bout trandeoffs, though I'm sure thye've heard it.

Lea: I don't recall the picture in picture case.. a lot of the discussion centered around chrome offering a download functionality for vidoes which made some content providers uncomfortable so they wanted to hide it so it's less prominent. The functionality is still there. Using controlsist to hid the download button, authors can still download the videos by right clicking. Whereas telling them we will not provide this functionality would mean they switch to another element. If it's more effort for the author so they ahve more incentive to use the native element... these days using a custom element is a oneliner, not extra effort. There are several legit use cases for customising the gui. Right now it's no controls at all, or all the controls the browser has to offer. With something like thhis they can display controls appropriate to use case - not all require every control. And if there is a way to customise what is displayed browsers have incentive to make new functionality available to authors, who can choose whether to use it, rather than browsers weighing whether it clutteres the control ui or not. My udnerstanding is push back comes from apple who do not implement a download functionality anyway. Not sure what the best approach is here.

Peter: a lot of sites will mask the right click behaviour to prevent people from doing a right click download> That feels like a sense of drm that is not a use case we should enable. There are cases with platform specific controls that may or may not show that web devs may not be thinking about - like air play. When I pull up a video I can't air play it to my tv

Tess: webkit default controls on apple platforms have an airplay button, we also expoes a js api, which ended up contributing designwise to a standardised api. We expose a js api so if you're building custom controls you can build your own airplay, so you have the ability to do that too.

Peter: very user hostile to hide controls that developers may not be aware of or thinking about that a user may find valuable. Developers are hiding them for aesthetic reasons

Tess: or capitalism reasons

Peter: I've turend of all controls and built my own because designer wanted a different look and feel

Lea: even though controls list is only supported in chrome, when we looked at actual usage ther ewas fairly significant useage on the web today, which does indicate user need

Tess: youtube may have adopted it

Lea: in terms of % of websites. Not just youtube.

Peter: that shows a developer need, not an end user need. Developer may be scratching an itch that is end-user hostile

Lea: sometimes you want a very small player and control which are the absolutely most important ones to display

Tes: yeah. Look at native controls in many browsers you might have a reduce set of controls visible depending on a bunch of factors

Lea: but browser doesnt know your use case - can only display what would work for some average

Dan: More what we need to do is write a document that captures this information, and post, or post a comment that says the essence of TAG discussion. We don't have consensus. We can see both sides. Sometimes it's okay to not have consensus. We can still point out issues like multi stakeholder.

Tess: yeah. Lay out pros and cons.

Dan: start a document? Progress it by plenary this week?

**we agree to

2022-06-06

Minutes

Tess: we should distill the tradeoffs ... however don't know if it would be useful. We could just say "we didn't come to consensus". "We're happy to see the complexity of the tradeoffs has been considered." I'm worried about them taking our output as a red light or green light. Something generic and short.

Tess: I will take the action to figure out that statement.

Dan: let's plan to review at the plenary and close if possible with that text.