#413: Screen Enumeration API
Discussions
Comment by @torgo Sep 10, 2019 (See Github)
Discussing in the f2f right now - @lknik can you take a look at the answers to the privacy & security questionnaire?
Comment by @hober Oct 15, 2019 (See Github)
Labeling as a Fugu-related request since it appears in Fugu's full list of capabilities.
Comment by @lknik Oct 18, 2019 (See Github)
Each of the new display properties chosen for this API makes a non-replaceable contribution. Removal or constraining of any property may degrade the user experience. Some properties offer more value than others, and some are more relevant to the proposed use cases than others. Care should be taken to choose those properties offering the most value to the most important use cases.
It has been suggested that user agents themselves could provide a UI for selecting where content is presented, but this is cumbersome for users and prohibitively limiting for web applications. This also offers no meaningful value over the existing cumbersome manual placement of windows by users.
Can't we simplify to "for each display we need same information as is currently offered by existing screen object"? We do not discuss the contents of the screen object, but the inflation of (many) screen objects.
2.3 This API does not expose such information. Any exposed information may identify the machine, but not its user.
Information allowing to identify the user's machine is more or less identifying the user, so you're saying the opposite. Maybe at least remove the 2nd sentence?
2.6 This API proposes exposing some additional properties for each of the displays connected to the device; see Section 2.1 for a complete list.
The complete list seems to be X*12-15 static objects (some of them dynamic; with X the number of screens).
2.7 Does this specification allow an origin access to sensors on a user’s device? No.
It does on the other hand inform if the display has an accelerometer, which would make it simpler to direct a malicious script to an appropriate window.
2.8 "The following display properties are currently exposed ..."
So we can today access all the device screens already? Wouldn't it be more useful to enumerate the new identifiers to be exposed, which is the above 12-15 numbers per each screen, which is not currently easily available?
To mitigate this issue, user permission is required to access the display
Do I see it correctly that this API will be gated behind permissions?
Comment by @hober Dec 3, 2019 (See Github)
Marking as pending external feedback
; @lknik‘s comments above have not been addressed, and we’ve also raised spark008/screen-enumeration#23.
Comment by @plinss Mar 2, 2020 (See Github)
Looked again during Wellington F2F, still waiting for external feedback. We also have concerns about how this intersects with ongoing work for foldable screens https://github.com/w3ctag/design-reviews/issues/472.
An additional observation is that there's an explicit non-goal of exposing all the OS screen APIs yet this proposal seems to do so.
Comment by @hober May 27, 2020 (See Github)
@spark008 I'm not sure if you saw @lknik's questions above. Could you take a look at them and get back to us?
Comment by @michaelwasserman May 27, 2020 (See Github)
Thank you for poking this thread and apologies for the delay! I'll try to address the open questions and feedback this week.
Comment by @michaelwasserman May 29, 2020 (See Github)
Thank you for your feedback and patience! I hope you find my responses and changes helpful, and I look forward to iterating further with additional input.
@hober thanks for your feedback and patience, I hope you find these responses helpful:
We also have concerns about how this intersects with ongoing work for foldable screens #472.
This work is complimentary to Foldable Screens #472. That proposal offers simple tools to support content spanning the single fold of a dual-screen device. Screen Enumeration and Window Placement (TAG review request imminent) aims to support a broader range of multi-screen and multi-window scenarios with lower-level information (eg. 3+ screens, 2 heterogeneous or misaligned screens, separate windows on separate screens, etc.).
An additional observation is that there's an explicit non-goal of exposing all the OS screen APIs yet this proposal seems to do so.
The goal is to expose a reasonable set of info for web applications to make informed use of the available screen space; that will require exposing some limited new information, but care has been taken to select the most valuable subset of available information. The spirit of the rephrased non-goal is to avoid exposing extraneous or especially identifying information (eg. EDID), and avoid exposing APIs to control the configuration of display devices.
we’ve also raised webscreens/screen-enumeration#23.
I posted a couple comments there, and want to explore ways to support requests for limited screen information. Please let me know if you have any thoughts regarding my comments there.
@lknik Thank you for your Security and Privacy review and comments, I hope you find the latest changes there and these responses helpful:
Each of the new display properties chosen for this API makes a non-replaceable contribution. Removal or constraining of any property may degrade the user experience. Some properties offer more value than others, and some are more relevant to the proposed use cases than others. Care should be taken to choose those properties offering the most value to the most important use cases.
It has been suggested that user agents themselves could provide a UI for selecting where content is presented, but this is cumbersome for users and prohibitively limiting for web applications. This also offers no meaningful value over the existing cumbersome manual placement of windows by users.
Can't we simplify to "for each display we need same information as is currently offered by existing screen object"? We do not discuss the contents of the screen object, but the inflation of (many) screen objects.
I modified the section to separate the goals of exposing multiple screens and new screen properties , both of which are valuable.
2.3 This API does not expose such information. Any exposed information may identify the machine, but not its user.
Information allowing to identify the user's machine is more or less identifying the user, so you're saying the opposite. Maybe at least remove the 2nd sentence?
You're right, I revised the section to accurately convey that new information is exposed that could be used to help identify a device, and mention the possibility to gate access upon user permission.
2.6 This API proposes exposing some additional properties for each of the displays connected to the device; see Section 2.1 for a complete list.
The complete list seems to be X*12-15 static objects (some of them dynamic; with X the number of screens).
I added a comparable summary in this section and added screen.[avail]top/left in section 2.1.
2.7 Does this specification allow an origin access to sensors on a user’s device? No.
It does on the other hand inform if the display has an accelerometer, which would make it simpler to direct a malicious script to an appropriate window.
I added a note to this effect in the section.
2.8 "The following display properties are currently exposed ..."
So we can today access all the device screens already? Wouldn't it be more useful to enumerate the new identifiers to be exposed, which is the above 12-15 numbers per each screen, which is not currently easily available?
I rewrote this section to give a more pointed answer to its question, including the difference of the single screen available to each window and the full set of screens, and a summary of what's currently available for each considered/proposed property.
To mitigate this issue, user permission is required to access the display
Do I see it correctly that this API will be gated behind permissions?
Yes, user permission should be a critical consideration of any browser that might adopt something like this proposal, and the asynchronous getScreens() should enable user-facing permission prompts.
Discussed
Jun 8, 2020 (See Github)
Tess: Looked at this in Wellington...
Peter: It was pending feedback, but we got some feedback.
Rossen: seems like they made quite a few updates.
Tess: Certainly seems like all issues were addressed, that's good.
(pausing for further investigation)
Comment by @michaelwasserman Jun 10, 2020 (See Github)
Please consider #522 alongside this proposal, they're highly related; thank you!
Comment by @michaelwasserman Jun 16, 2020 (See Github)
You may wish to consider these specific questions during review. Thank you!
- Reuse existing Screen interface or add something new? webscreens/screen-enumeration#31
- API design for a single multi-screen boolean webscreens/screen-enumeration#30
- Combine Screen Enumeration and Window Placement proposals? webscreens/screen-enumeration#34
- Reject the getScreens() Promise with permission denial webscreens/screen-enumeration#29
- Scope of the initial proposal webscreens/window-placement#15
Discussed
Jul 20, 2020 (See Github)
Bumped to next week
Discussed
Jul 27, 2020 (See Github)
Ken: They are asking for feedback on 5 things (last comment in issue). Like should we add to existing screen in addition to getScreens() or should we just add the new props to the screen objects returned by getScreens().
Ken: navigator.screen is the main screen. they want to add additional things (beyond what's already defined in CSSOM View). [list of many features] They want getScreens() in addition because you may have more than one; should the new things just be added to the new objects or should they also get added to navigator.screen
Tess: shouldn't they be the same type of Screen object?
Ken: the new things are async, and they want a static snapshot instead of having a live object. Also the current object is sync and permissionless.
Peter: my initial reaction was similar to yours. But now I think navigator.screen is a mistake. We should deprecate it over time. Maybe the new Screen object should have a similar API surface, but be distinct. I think we should pretend navigator.screen doesn't exist and start over.
David: Do all of the things on the existing screen object even make sense for all screens, or just the main screen? [examples] spec, Gecko IDL, Blink IDL
Tess: should we file CSSOM View issues to ask them to deprecate the main screen object, and cut its properties down to just what is necessary for web compat?
Ken: That sounds good
Peter: availTop
and availLeft
make sense if you have top
and left
, but those only make sense if the geometric relationship between all the screen is known. (I suppose it is, but can change over time.)
Ken: [takes action to leave comment to that effect]
David: we shouldn't weigh in too strongly, in case they have good reason to do something different.
Ken: the next one is whether getScreens() should support filtering, perhaps with media queries
Tess: We have some advice in our design principles document about this kind of device enumeration. We recommend providing the ability to filter or constrain the devices that get returned.
Tess: They shouldn't rely on a list that provides all the screens.
Peter: not sure the multiscreen boolean makes sense.
Tess: They should also rigourously spec the order in which screens are returned (per our design principles)
Ken: [leaves comment summarizing the above]
Tess: [time check]
Peter: third issue is #34 combining screen enumeration and window placement proposals
Tess: argument for is that they need to be used together to solve the use cases, and argument against is theoretical purity? they're logically distinct?
Peter: relationship between screens can change over time, but that's not related to this issue. this is the relationship between windows and screens.
Tess: on the one hand, I like reviewing short specs that define one, self-contained feature. On the other hand, I tend to prefer kitchen sink specs (like HTML) over lots of little specs (like CSS). I think it makes editors think through how things are interrelated more. All that said, I think this question is ultimately one of editorial discretion. My quirky preferences shouldn't outweigh the preferences of the people actually doing the work.
Tess: I will take the action item to give the feedback
Tess: issue #29 now, reject promise with permission denial
David: Seems reasonable to reject the promise.
Ken: As we suggested deprecating navigator.screen it would be bad if you can not get the main screen without a rejection/permission
David: Not sure this should be behind a permission and instead be browsers settings etc.
David: I will leave some feedback. Done.
Ken: last question is about scope
David: seems complicated; different OSes and different browsers do different things here.
Tess: regarding #15 scope, I encourage them to take baby steps - I will leave feedback.
Comment by @hober Jul 27, 2020 (See Github)
We talked about this during a breakout today. We'll leave comments on each of these issues.
Comment by @michaelwasserman Jul 31, 2020 (See Github)
I believe I have addressed all recent feedback, thank you!
Discussed
Aug 24, 2020 (See Github)
Ken: specific issues - we gave feedback - they say they have addressed our feedback.
Tess: issues were auto-closed?
[looking at their issues to see if the feedback has been addressed]
Ken: I'm fine with this... One of them they have filed an issue which hasn't been fixed yet.
Dan: are there key issues that haven't been resolved - or can we close?
Ken: for promises they have listened, for scope they are considering, and they have combined screen enumeration and placement which also follows our feedback so I would say yes. And they filed new issues in their new repo.
Dan: Propose we close then.
Peter: my main concern is the inetraction with the foldables work.
Ken: thomas steiner filed an issue related to that issue 21 on the new repo... Nothing there yet but the issue is filed.
Peter: my main fear is that - the foldables work is in flux - someone's going to ship something and then it will be an API we can't change and it won't work nicely with foldables...
Ken: i think that's why they asked about the scope.
Dan: we could leave that specific feedback - that this needs to play well with the developing foldables work.
Peter: i'm fine with that.
[consensus to close]
Ken: I can [leave a closing comment and close it
Comment by @kenchris Aug 24, 2020 (See Github)
Hi there,
We are fine with the work you have done and we are therefore going to close this issue. @plinss is still a bit worried that part of this might ship in a way that it is incompatible with the work around foldables, and will file an issue about that on your new repo.
Thanks for staying at home with the TAG!
OpenedSep 3, 2019
こんにちはTAG!
I'm requesting a TAG review of:
Further details:
We recommend the explainer to be in Markdown. On top of the usual information expected in the explainer, it is strongly recommended to add:
You should also know that...
This API is intended to be used in conjunction with the Window Placement API, which is still in the proposal stage.
We'd prefer the TAG provide feedback as (please select one):
Please preview the issue and check that the links work before submitting. In particular, if anything links to a URL which requires authentication (e.g. Google document), please make sure anyone with the link can access the document.
¹ For background, see our explanation of how to write a good explainer.