#679: Conditional Focus (When Display-Capture Starts)

Visit on Github.

Opened Sep 22, 2021

Braw mornin' TAG!

I'm requesting a TAG review of Conditional Focus.

When an application starts capturing a display-surface, the user agent faces a decision - should the captured display-surface be brought to the forefront, or should the capturing application retain focus.

The user agent is mostly agnostic of the nature of the capturing and captured applications, and is therefore ill-positioned to make an informed decision.

In contrast, the capturing application is familiar with its own properties, and is therefore better suited to make this decision. Moreover, by reading displaySurface and/or using Capture Handle, the capturing application can learn about the captured display-surface, driving an even more informed decision.

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • Relevant time constraints or deadlines: N/A
  • The group where the work on this specification is currently being done: WebRTC WG
  • The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): WebRTC WG and/or WICG
  • Major unresolved issues with or opposition to this specification: N/A
  • This work is being funded by: Google

You should also know that...

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify @eladalon1983

Discussions

Discussed Sep 27, 2021 (See Github)

Amy: it's in a private repo

Dan: if this is part of webrtc why isn't it going on in webrtc wg

Amy: It's in WICG now. And discussion in webrtc - long and active, looks like detailed technical discussion. Odd they're asking for TAG review if they're still working out details?

Dan:

Hi @eladalon1983 - Looking at the [thread in WebRTC](https://github.com/w3c/mediacapture-screen-share/issues/190), it looks like this is still under active discussion. If so, it seems like it might be premature for the TAG to be reviewing this?  Is the plan to incubate this in WICG and then bring it to WebRTC for a future WebRTC rechartering?

Amy: short security/privacy stuff.. can dig in and see if there's anything else we should flag on that.

Dan: I'll assign us

Sangwhan: mismatch between different browser vendors on how many people work on webrtc, causes interop issues

Comment by @torgo Sep 28, 2021 (See Github)

Hi @eladalon1983 - Looking at the thread in WebRTC, it looks like this is still under active discussion. If so, it seems like it might be premature for the TAG to be reviewing this? Is the plan to incubate this in WICG and then bring it to WebRTC for a future WebRTC rechartering?

Discussed Dec 1, 2021 (See Github)

Amy: all of the use cases are from the perspective of the user agent... the browser needs to x, etc... no privacy & security review... About the browser deciding what needs to be in focus... They've not explained why it might benefit the user... One security concern about clickjacking...

Hello, @torgo and I looking at this in our virtual face-to-face today. We noticed the problem statement in the explainer is from the perspective of the user agent, rather than the end user. Would you mind adding some examples/use cases/scenarios of how the end user of an application benefits or is affected by this feature? We appreciate that you have noted one security concern in the explainer; do you have an idea of when you will be able to complete the [privacy and security questionnaire](https://www.w3.org/TR/security-privacy-questionnaire/)? Are you expecting to take this work to the WebRTC WG?
Comment by @rhiaro Dec 7, 2021 (See Github)

Hello, @torgo and I are looking at this in our virtual face-to-face today. We noticed the problem statement in the explainer is from the perspective of the user agent, rather than the end user. Would you mind adding some examples/use cases/scenarios of how the end user of an application benefits or is affected by this feature? We appreciate that you have noted one security concern in the explainer; do you have an idea of when you will be able to complete the privacy and security questionnaire? Are you expecting to take this work to the WebRTC WG?

Comment by @eladalon1983 Dec 8, 2021 (See Github)

Hi @eladalon1983 - Looking at the thread in WebRTC, it looks like this is still under active discussion. If so, it seems like it might be premature for the TAG to be reviewing this? Is the plan to incubate this in WICG and then bring it to WebRTC for a future WebRTC rechartering?

It's being incubated in the WICG at the same time as it's presented to the WebRTC WG. (It's also in origin trial in Chrome starting m96.) We've had some fruitful discussions in the WebRTC WG and I am hoping we'll reach a mutually agreed-upon API shape in January or possibly earlier.

Hello, @torgo and I are looking at this in our virtual face-to-face today. We noticed the problem statement in the explainer is from the perspective of the user agent, rather than the end user. Would you mind adding some examples/use cases/scenarios of how the end user of an application benefits or is affected by this feature?

I can rephrase this with the next revision, no problem. Could you share more of your thoughts about the phrasing? For example, I took a look at the Web Cryptography API and it looked to me to also be centered around the user agent and the applications, not the user. The way I see it, in both cases, the benefit to users is applications can serve them better. In the case of Conditional Focus, this better service is through less distractions thrown the user's way, and less clicks required of the user, when they are engaged in something as mentally-challenging as presenting to a live audience. Would adding an explicit shout-out of this fact at the end of the Problem Description section be sufficient?

We appreciate that you have noted one security concern in the explainer; do you have an idea of when you will be able to complete the privacy and security questionnaire? Are you expecting to take this work to the WebRTC WG?

Presently, I expect to do that after the final shape is agreed upon in WebRTC WG discussions.

Comment by @rhiaro Dec 8, 2021 (See Github)

this better service is through less distractions thrown the user's way, and less clicks required of the user, when they are engaged in something as mentally-challenging as presenting to a live audience

Thanks, that's exactly the kind of language we like to see in an explainer. It can also be helpful to walk us through how someone interacts with the feature from first principles, assuming little prior familiarity with the details of the work. (If you haven't already seen it, there are some links to good explainers in the explainer explainer.)

Discussed Jan 31, 2022 (See Github)

Amy: we asked for updated explainer and privacy & security. No update yet. They said they hope to reach an agreed api shape in january or earlier with webrtc wg so we could ask about that

Dan: [writes comment]

Sangwhan: trying the demo, it makes perfect sense what they're doing here

Comment by @torgo Feb 1, 2022 (See Github)

Hi @eladalon1983 - can you let us know status on the API shape (and security & privacy questionnaire response)? Also have you thought any more about documenting the user need - rephrasing and expanding the use case from an end-user perspective - as @rhiaro mentioned above?

Comment by @eladalon1983 Feb 22, 2022 (See Github)

(Apologies for still not responding, but I am prioritizing other work. Please leave this open. I intend to get back to this in the next few weeks.)

Discussed Apr 4, 2022 (See Github)

Dan: on 22nd feb said they'll get back in the next few weeks, haven't heard. Ping again?

Sangwhan: I don't think this has been worked on recently

Comment by @torgo Apr 5, 2022 (See Github)

Hi @eladalon1983 just pinging on this. Has there been any update we should be aware of?

Comment by @eladalon1983 Apr 5, 2022 (See Github)

Sorry, still in the pipeline.

Discussed Apr 18, 2022 (See Github)

bumped 3 weeks as Elad got back to us and said it was still in the pipeline.

Discussed Jun 27, 2022 (See Github)

Amy: seems stalled by the requester

Dan: closing for now, can reopen when it's ready for review

closed

Comment by @torgo Jun 28, 2022 (See Github)

Hi @eladalon1983 we're going to close this for now just to keep our agendas in order. Can you ping us when you're ready for us to re-open and review?

Comment by @eladalon1983 Jun 30, 2022 (See Github)

Will do.

Comment by @eladalon1983 Oct 13, 2022 (See Github)

Can you ping us when you're ready for us to re-open and review?

Following discussions in the WebRTC WG, the API was reshaped. The new shape is visible in these PRs:

I'll be updating the explainer soon. Please take a look.