FYI for @plinss and @dbaron that this got another ping today. I know we discussed the media query events as a way to detect when this changes, but not sure if we resolved one way or the other if the API provides enough support to ensure that a developer can actually respond correctly to a window being dragged from one window w/o wide gamut to another tha does.

Thoughts?

It's rather hard to review this as the formal description isn't really there yet (as I also pointed out in the HTML PRs).

I summarized the surface changes of this spec here: https://github.com/whatwg/html/issues/4167

A few questions to start:

• how does this relate to the wide-gamut work happening in CSS (primarily in css-color-4, I think)? See in particular the section on Profiled, Device-dependent Colors (although I'm really not a fan of that section name). /cc @svgeesus @grorg @smfr
• what happens with color spaces here? It seems like that's an essential part of the problem space given that it's my understanding that sRGB isn't well-defined outside of [0, 1]. It seems to be discussed in the explainer but not in the proposed spec changes.
• how does this relate to capabilities of the OS or device? @slightlyoff alludes to that above, but I don't see the relationship explained in the explainer.
• how do the proposed solutions relate to the three use cases given at the top of the explainer? And what application use cases benefit from those (still mostly very technical) use cases? Are the proposed solutions sufficient for those applications?
• the explainer says that 12 bits is needed for rec2020, but then suggests float16, which has 11 bits of precision. Is that good enough?

We are not releasing the full WICG proposal, only the wide-gamut part of it. When in wide-gamut the color space defaults to extended sRGB.

The term extended sRGB is not defined, or even used, in the explainer. As @dbaron correctly notes, the sRGB transfer function is undefined outside [0, 1] and there are multiple plausible ways of extending it, so at a minimum that needs to be fully defined. Having done so, the capability of the extended space to represent WGB (and indeed, HDR) can be evaluated.

The explainer merely notes that 8bit " is below the capabilities of some monitors" which is true, but fails to connect that statement to why 10 or 12 bits is needed:

• to represent wider gamuts (or wider dynamic range) without increasing the size of steps between adjacent color values (which would give banding, etc)
• to allow correction of tone curves for calibration and linearization (this is why some monitors have, for example, 14bit lookup tables for gamma correction, even when the inputs are 8,10 or 12bit)

@dbaron makes the good point that Rec.2020 uses 12bits (in fact either 10 or 12, UHD uses 10 with 2020) while the proposed 16-bit half-float has 11 effective bits. That might well be okay, but it would be good to know that half-float is not painting us into a corner.

The Summary of the wide gamut support on Canvas proposal says

enum ImageDataStorageType {
  "uint8", // default
  "uint16",
  "float32",
};

The float16 option is missing there. Or is it being stored as a union inside the uint16?

@svgeesus, regarding the float16 option. The backing storage is float16, but the output to javascript (on ImageData) is either uint16 or float32.

I agree we need a better stable definition of extended sRGB. I'm going to look into that and report back.

Here's a blub defining Extended-sRGB.

sRGB

Let's start with the familiar definition of sRGB (from Wikipedia), and then I'll introduce extended-sRGB.

And let's just look at the forward transformation (from CIE XYZ to sRGB). The reverse transformation is just the inverse.

We start with a triple of reals [X,Y,Z] in CIE XYZ space, and the output is a triple of reals [R,G,B] in sRGB space. There are two steps to the transformation. The output of the first step is a triple [Rlinear,Glinear,BLinear].

CIE XYZ to sRGB Step 1

The first step is defined as:

    [ Rlinear ]   [  3.2 -1.5 -0.5 ][ X ]
    [ Glinear ] = [ -1.0  1.9  0.0 ][ Y ]
    [ Blinear ]   [  0.1 -0.2  1.1 ][ Z ]

CIE XYZ to sRGB Step 2

For the second step, we define the function T (taking reals to reals) as follows:

           { 0               : x < 0
    T(x) = { 12.92*x         : 0 <= x <= 0.00313
           { 1.055*x^(1/2.4) : 0.00313 < x <= 1
           { 1               : x > 1

The second step and final step in the transformation is then

    [ R ]   [ T(Rlinear) ]
    [ G ] = [ T(Glinear) ]
    [ B ]   [ T(Blinear) ]

Remarks

That's it. Note that the function T only outputs values in the range [0,1].

Many applications encode the range [0,1] as 8-bit fixed-point.

Extended-sRGB

As before, we start with a triple of reals [X,Y,Z] in CIE XYZ space, and the output is a triple of reals [R,G,B] this time in Extended sRGB space. There are two steps to the transformation. The output of the first step is a triple [Rlinear,Glinear,BLinear].

CIE XYZ to Extended-sRGB Step 1

This is idential to CIE XYZ to sRGB.

CIE XYZ to Extended-sRGB Step 2

For the second step, we define the function U (taking reals to reals) as follows:

        { 1.055*(-x)^(1/2.4) : x < -0.00313
 U(x) = { 12.92*x            : -0.00313 <= x <= 0.00313
        { 1.055*x^(1/2.4)    : x > 0.00313

Note that in the range [-1,1], U is a point-symmetric extension of T, that is, U(x) == sign(x)*T(abs(x)).

The second step and final step in the transformation is then

 [ R ]   [ U(Rlinear) ]
 [ G ] = [ U(Glinear) ]
 [ B ]   [ U(Blinear) ]

Remarks

Note that it is entirely possible for R, G, and B to be negative or to be arbitrarily large (greater than 1). While these values may not correspond to physically realizable light spectra, it is valid to do math on them (just like in the CIE XYZ space), and once they are to be displayed in a particular gamut, the values may be clipped to that gamut.

Reverse transformation

The reverse transformation from Extended-sRGB to CIE XYZ is the inverse function, as with sRGB. Unlike with sRGB, the forward and reverse transformation are 1-to-1.

Remarks about compatibility

Note that for any triple [R,G,B] with R,G,B in the interval (0,1), the transformation of [R,G,B] from sRGB to CIE XYZ is exactly the same as the transformation from Extended-sRGB to CIE XYZ. This means that any program that was written in the sRGB color space can be changed to use the Extended-sRGB color space with no extra modifications.

Note that when using Extended-sRGB it is natural to store the color values as floating-point (or half-float) triples. It is not reasonable to encode them as 8-bit fixed-point values. If a canvas that is Extended-sRGB is to be converted to an image for export, then it would be natural to

• First convert from Extended-sRGB to the desired output image gamut (in the case of sRGB, this is the identity)
• Then clamp to the range [0,1] and encode as the range [0,1] as 8-bit fixed-point.

OK so of the three ways to do it:

a) extend sRGB function by not clamping to [0,1] b) extend sRGB function by mirroring about 0 c) use a linear function, and more bits

this proposal is using b). This means it is well defined above 1 and below 0, can represent wider gamut colors that require a negative component value (typically red), and can represent a modest increase in maximum linear component value (the curve flattens off after 1). This is probably enough to deal with WCG and may be enough to deal with HDR.

Please put that description into the proposal.

This means that any program that was written in the sRGB color space can be changed to use the Extended-sRGB color space with no extra modifications.

With few modifications:

• remove any clamping of out of range colors
• use a higher bit depth

While these values may not correspond to physically realizable light spectra,

true, but not a problem

it is valid to do math on them (just like in the CIE XYZ space)

for some definition of valid. For example, unlike XYZ or linear sRGB, , this is not a linear-light space. So computing, for example, the color that results from mixing two lights of different colors will not give the correct result. (It may still be useful to do, especially if performance is more important than accuracy, but note that the errors are significant and certainly visible).

For example, unlike XYZ or linear sRGB, , this is not a linear-light space. So computing, for example, the color that results from mixing two lights of different colors will not give the correct result. (It may still be useful to do, especially if performance is more important than accuracy, but note that the errors are significant and certainly visible).

Yes, we spent a while (independently on several different occasions) trying to push users to use a physically linear space, but that ended up backfiring on us. There's no one thing that people consistently want ... I found that

• Users (in my experience) want physical linear interpolation for...
  • anti-aliasing
  • decimation/magnification of images
  • physically-based rendering
• Users (in my experience) want perceptually linear interpolation for...
  • gradients
  • alpha blending transparency And of note is that sRGB (and Extended-sRGB) are ... closer ... to perceptually linear than physically linear (if one doesn't change color hue it is pretty close to perceptually linear ... if one does change hue, horribleness results).

This is probably enough to deal with WCG and may be enough to deal with HDR.

We've done some experiments with the Windows 10 HDR support (on HDR TVs) in WebGL, and the results have been promising!

While these values may not correspond to physically realizable light spectra,

true, but not a problem

it is valid to do math on them (just like in the CIE XYZ space)

for some definition of valid. For example, unlike XYZ or linear sRGB, , this is not a linear-light space. So computing, for example, the color that results from mixing two lights of different colors will not give the correct result. (It may still be useful to do, especially if performance is more important than accuracy, but note that the errors are significant and certainly visible).

@ccameron-chromium I agree that there are use cases for both. It is also true that the color space in which something is specified and the color space in which it is best manipulated (interpolation, blending etc being just one type of manipulation) need not be the same.

If you have a pointer to any write up of your experiences with HDR, I would be interested to read it.

Taken up at Tokyo f2f 2019.

The explainer appears to be shaping up nicely. There a number of issues discussed above that need to be more fully fleshed out. Our understanding is that the right people are currently connected in the discussion, so we're confident that good progress will be made there.

One general concern we have is that there are many areas of the platform that will be impacted by these changes, and we want to be sure that they are all being considered. (We don't have any immediate cause for concern, however.)

We'll keep this issue open to track progress on the Explainer as well as more detail being added to the other relevant specifications. Thanks for filing the review!

Hi,

It looks like the explainer is a little out of date; I think the current API proposal is captured in https://github.com/whatwg/html/issues/4167.

Could the explainer be updated to match the most recent proposal?

It seems likely that this will end up requiring various interfaces that expose canvas data to Javascript to expose the float16-based data to JS. It seems likely that defining that will require that WebIDL have a float16 or similar type that defines what the clamping behavior is. (This will then allow it to be consistent between parts of the Web platform and between implementations.) Is my assumption that this will be needed correct... and if so, is there an issue on file against WebIDL?

@hober and I just looked at this in a breakout at the Reykjavík face-to-face.

We think this is looking reasonable, and relevant experts seem to be involved in the discussion in whatwg/html#4167 (though it's not entirely clear how fast things are progressing there). We encourage you keep the explainer up-to-date as the specification evolves (see comment above; we definitely noticed a difference between "8-8-8-8" and "uint8"), and we hope that you can file the above issue against WebIDL if it's necessary.

I don't think we see the need to keep this issue open, although this is the sort of area that requires some attention to making sure there aren't missing pieces across the far-flung parts of the platform that require changing. Please let us know if you think further review is needed, either because there are substantial changes to the proposal or because you detect further areas where you think we need to look at how pieces in different parts of the platform fit together (which are easy to miss).

Thanks for submitting this request for review.