I'm requesting a TAG review of the Viewport Segments Property after taking into account feedback (e.g. webscreens/window-segments#12) around integrating our original Window Segments API proposal (initial TAG review here: #492) with the Visual Viewport API.
Web developers targeting foldable devices want to be able to effectively lay out the content in a window that spans multiple displays. However, the web platform does not yet provide the necessary primitives for building layouts that are optimized for foldable experiences. The Viewport Segments Property will provide a list of logical segments of the viewport that web developers can use to target with different views, as appropriate based on size and configuration of those segments.
I am very happy to see this being part of VisualViewport now. I like the figures/images from the explainer and think it would be nice to add those to the spec as well
It seems like this API might expose hardware characteristics to the web. Would it make sense to add text to the explainer's "Security & Privacy" section discussing how identifying these characteristics are, quantifying the fingerprinting risks, whether they're exposed by other APIs, and what mitigations you've considered?
@kenchris thanks! I can take an action item to get those added.
@JensenPaul, we've called out the additional bits of entropy and the fact that for some devices they're already exposed via window.screen properties in the questionnaire. The "Security and Privacy" sections of each explainer (I figured I'd comment here to address your comment in #690 as well) detail the mitigations we've put in place for each new primitive.
On devices with an actual foldable display, it makes a lot of sense to talk about segments when folded, with the addition that the segment in the middle (the actual folded part) should also be exposed as it might not be touchable (or touch events might not work well). @darktears @diekus @lauramorinigo
The reason I am asking is that it would be nice to discuss and show example of this in the explainer/spec. Like I don't know what the current thinking is for foldable displays. Will it just expose 3 regions, with no overlap (first, fold, last), or will the fold area overlap with the first and last region? Would all of this work with this API?
Some gamer laptops have 1.5 screen, basically a mini display above the keyboard that can be "lifted up" together with the main display to become a larger "foldable" monitor. You could imagine the lower display (and thus segment) to support touch, but not the main screen (other segment) given the gamer focus (cost saving) - I understand that on systems today these are considered different monitors, but AFAIU most of that was done due to missing platform APIs and this could change in the future.
Can we clarify in the explainer that this doesn't cover regular multi monitor use-cases and only joined displays that are considered one virtual display/screen?
In our TAG discussions we always end up with confusion around virtual displays (what the OS considers a display) that might actually be multiple physical joined displays. Can we please clarify that in the explainer and spec by linking to the relevant parts in https://webscreens.github.io/form-factors/
@kenchris In regards to your comment on foldables. With the refactor to support N hinges/folds it's getting harder to support that because there is no more information about the hinge/fold area. The segments property as described here will return two segments on a dual screen device where the geometries of each will allow you to layout around the hinge. Just like the https://drafts.csswg.org/css-env-1/#viewport-segments. I agree with your comment that if a foldable/OS decide to divide the viewport in two when the device is half folded, we could return three segments (two for the top/bottom and one for the fold area). Then it's up to the developer to decide whether to use a split UX or not. But in this case the "fold" segment must be defined to let the developer know where it is because of usability reasons (for e.g. avoiding putting a button or a scrubbing bar in the fold area) regardless of split UX.
I asked Microsoft in the past to add a property that would let a developer identify if the hinge is solid or transparent justly to cover that. Thinking about this proposal we'll have to expose the segments into an object that contains the DOMRect and the type (standard segment vs fold(opaque)/hinge(solid).
The fold case is something we should solve here and in the CSS counterpart. Our partners were specific around some of the issues with foldables: half cut videos, buttons in the middle of the fold are challenges that you see often. We need developers have the tools to address the issue.
Dan: I know there are multiple implementations. But all in Chromium. Different stakeholders are interested. Looking at S&P.. don't think theyre exposing any extra info. I think this is good, but I've been kind of working on it.
Peter: one concern is that the segments array is just a bunch of DOM rects. Comments between Ken and darktears that maybe it should have some embedded... should return objects, so room for expansion. Can leave comment. Other concern in general is that they seem to degrade very quickly into only dealing with one particular hardware situation, eg. screens are all the same size, or only two segments. This one doesn't suffer from that, the segments could be anywhere in space. No information aside from doing math with rectangles to figure out hwo they're oriented. Maybe there's metadata thtat can be added later.
Yves: related to second screen?
Dan: no
Peter: second screen is like casting. This is multiple monitors or foldable screens.
Dan: a few different hareware configurations this makes sense for. Gets very complicated if you start to hink about screens with multiple hinges.
Peter: I'm right now sitting in front of 3 monitors. There are existing groups of things that aren't just a foldable phone. I want it to not be treated as something different from a foldable phone. I think this one API is fine for that.
Dan: design is kind of born out of that concern. Intended to address that concern, to make sure whatever we do here is forward looking. Look again at plenary.
Rossen: both (689 & 690) are being experimented with
Dan: Peter asked a question about object vs DOM rect. No response yet.
Rossen: what would be the lifetime of such object? The beauty of rects is that they're immutable valid for the time being. If I return object now we're talking about bringing in the whole kitchen around the sink with events..
Peter: doens't need an event, doesn't need to be live, could be a snapshot. This isn't a blocker. Just a question. Negotiating API shape, overall happy with the feature.
Dan: Close with that question? Ping requester again?
Peter: worth completing the conversation
Dan: could close on wednesday if we get that feedback
Hi @dlibby, @ststimac, @scottlow... we're trying to progress this review so we can get you some useful feedback and close it. Do you have any response to Peter's question from earlier in the year? Can anyone give us an update on the evolution of this proposal? Also where is this going after WICG? Any news there?
Hola @torgo, 2 out of 3 members there are not in the team anymore. Let me follow up internally and get you some much needed peace. Will report back soon.
Adding an extensibility point via a more complex object sounds like a reasonable way to stage this API. But I'm not sure I'm convinced unconditionally exposing the hinge as a segment is a net positive for developers, and either muddies the waters w.r.t. the concept of viewport segments as exposed via environment variables or makes them much more complex to use. Designing for future form factors is one thing, but consider the real-world devices in market today - Samsung foldables have no hinge, while Surface Duo does (i.e. the hinge consumes part of the viewport). Do we really want developers to have to add if statements and duplicate logic to support separate dual-view implementations on those devices?
Thanks folks - on the basis of this feedback we're happy to close this and we're happy to see this work move forward. Looking forward to a foldable future.
Thanks folks - on the basis of this feedback we're happy to close this and we're happy to see this work move forward. Looking forward to a foldable future.
OpenedNov 9, 2021
Braw mornin' TAG!
I'm requesting a TAG review of the Viewport Segments Property after taking into account feedback (e.g. webscreens/window-segments#12) around integrating our original Window Segments API proposal (initial TAG review here: #492) with the Visual Viewport API.
Web developers targeting foldable devices want to be able to effectively lay out the content in a window that spans multiple displays. However, the web platform does not yet provide the necessary primitives for building layouts that are optimized for foldable experiences. The Viewport Segments Property will provide a list of logical segments of the viewport that web developers can use to target with different views, as appropriate based on size and configuration of those segments.
Further details:
We'd prefer the TAG provide feedback as:
💬 leave review feedback as a comment in this issue and @-notify [github usernames]