Hi,

We had an initial look during 10/07/19 telecon.

What we (and myself) are wondering are:

1. This looks like an interesting metric, and potentially something developers would choose to optimise for. Is it a good idea to focus optimisations to improve this particular score? Any particular unobvious concerns come to your mind?

2. Any general estimations on the score variations? How stable is the metric? Would it be particularly affected by dynamic elements like fonts?

3. How frequently is the score updated? At any change? Would that add to any further performance impacts?

4. Lastly: the metric would depend on the engine, the CPU, the network, etc. So site and possibly also user dependant. While access is to be "limited" to the particular site only, but can we reasonably imagine an "adversarial" design that would be crafted in a way to extract something attributable to the user's computing setting across sites?

Thanks for the feedback!

We see this metric as part of a set of metrics, alongside First Input Delay (FID) and First Contentful Paint (FCP), that provide insight into user experience. Developers should not optimize for any single metric to the exclusion of others, but should optimize for all of them together.

We have found that web fonts are a common source of layout instability. I think platform improvements are needed to support web fonts with stable layout. There has been some discussion of this, e.g. on crbug.com/898083. The score can also vary with things like ad selection.

The layout shift score is generated for every rendering update that contains layout shifts. These updates are throttled to the speed of animation frames (vsync). There could still be a performance impact if the developer does something expensive in the PerformanceObserver callback.

A website has many ways to deduce information about the user's computing capabilities for purposes such as fingerprinting. The layout instability metric is one such way, but we don't think it exposes information that couldn't be just as easily obtained through other APIs that measure the speed or query the state of layout.

Hi,

Thanks for your input. Here I'm only going to refer to the last point.

A website has many ways to deduce information about the user's computing capabilities for purposes such as fingerprinting. The layout instability metric is one such way, but we don't think it exposes information that couldn't be just as easily obtained through other APIs that measure the speed or query the state of layout.

I'm not suggesting there are any unprecedented new risks or methods of using that, so please don't get me wrong. Still I would suggested in thinking in these ways:

• what does it allow/enable?
• is it possible to do this now, if so with what and how?

Would be great to have it documented.

Also last and not least, I believe that the score value is double. In your view in practice would be what scale [0,x] in practice (or theory)? Can it be bounded?

These are good questions to think about. It's already possible for a site to query an element's bounding rect, but it is infeasible to query every element at every point in time. So I would say the metric enables observing layout changes in a way that isn't efficient today.

The explainer contains some comments under "Privacy and Security" about the possible use of the API for fingerprinting, and about cross-origin subframe restrictions which the API respects.

If you have other ideas about unexpected capabilities the API might enable I would be interested in hearing about them.

I think a rough guideline on scoring might be 0.0 - 0.1 = low jank 0.1 - 0.5 = medium jank 0.5 - 1.0 = high jank

Scores above 1.0 are possible but the differences become less meaningful. (The highest scores tend to be sessions with auto-repeating layout-inducing animations.)

I expect we'll refine our understanding of this over time based on the data we observe in the wild.

We're wondering what the effective precision of this metric is, given the guidelines you outline above and this comment in the explainer:

It is intended that the LS score have a correspondence to the perceptual severity of the instability, but not that all user agents produce exactly the same LS scores for a given page.

If that is the case, would it make sense to expose this as, say, an integer between 1 and 10 (assuming variance within those 10 buckets is essentially noise), and truncate values higher than 10 to 10?

So another question here: it seems like figuring out what developers should optimize for in terms of stability during load is a question that has active research. At least, I don't think there's an agreed-upon answer for what the perfect thing to measure is.

So for this specification, it feels like there's a tension between (a) specifying something that's precisely interoperable versus (b) specifying something that can be improved on over time (e.g., where browser engines could have different opinions on what type of instability is best/worst for their users, and could improve that metric over time).

I'm curious both where you think the current spec fits along that tradeoff/continuum, and what you think advantages/disadvantages of shifting to a different point along that continuum would be.

Also, is there a plan to move this work to the Web Performance WG at some point?

Thanks for your answers @skobes!

These are good questions to think about. It's already possible for a site to query an element's bounding rect, but it is infeasible to query every element at every point in time. So I would say the metric enables observing layout changes in a way that isn't efficient today.

So it seems to me it is in fact introducing something that is not possible today (it might be possible, maybe, with different approach, less efficiently, etc?) after all. Why not document this down?

Any view on (ways of) manually introducing element jitter? What that be seen by this metric?

I think a rough guideline on scoring might be 0.0 - 0.1 = low jank 0.1 - 0.5 = medium jank 0.5 - 1.0 = high jank

Scores above 1.0 are possible but the differences become less meaningful. (The highest scores tend to be sessions with auto-repeating layout-inducing animations.)

Looks good. Would it be possible to reduce values>1 to 1, and perhaps even keep just 0, 0.2, 0.5, 0.7, 0.9, 1? Alternatively 0.{1,2,3,4,5,6,7,8,9}, 1.

I expect we'll refine our understanding of this over time based on the data we observe in the wild.

How long would this observation phase last?

There's a bunch of useful feedback in @frivoal's post to blink-dev.

Parts of @frivoal's post appear to be covered by WICG/layout-instability#19, WICG/layout-instability#20, and WICG/layout-instability#21.

So it seems to me it is in fact introducing something that is not possible today (it might be possible, maybe, with different approach, less efficiently, etc?) after all. Why not document this down?

I think it probably is possible today by more directly observing the things that cause the layout instability: put a bunch of load events on images and fonts, record the times from them, also record the times from any key points in script execution that would modify the layout, etc. This would more directly record the performance measurements than the layout instability score, which would really be a lower-information proxy for those other things.

We've filed a few issues (see links above), and I think we're reasonably happy with the responses so far. Some of those issues do reflect things that we'd like to see documented better in the spec, but we don't think we need to keep this review open for that.

This also isn't a particularly detailed review; we hope that as this spec stabilizes it moves into a forum (most likely the Web Performance WG) where it can get appropriate detail-level review from other potential implementors.

So thanks for requesting TAG review.