#951: Conversion to RGB in VideoFrame.copyTo()
Discussions
2024-09-02
Question open to Lea about color spaces stuff in CSS. Architectural desire to ensure that color is treated consistently across the platform.
This really needs to be taken to the Media WG. A gist is not really good enough.
Is the name "copyTo" the right name? Or is this a color space transform? Are there other transforms that the platform should support? See The Design Principle issue on naming.
2024-09-09
Martin: I think this is satisfied but there is a need for greater coordination across the platform for how color is handled... The CSS area is more advanced with color spaces, the media pipeline is more advanced with HDR. They're not talking to each other.
Yves: name might not be right ... because it's doing a conversion to RGB... instead of just copying to something...
Thanks for giving this opportunity to review this work. We see this as a useful addition to the platform.
We do observe that there is a lot of uncoordinated work on color across the platform. Our understanding of the situation is that CSS is quite advanced with its work on color spaces and the media pipeline (video and so forth) is more advanced with high dynamic range content. Having a uniform platform-wide would be desirable and we'd encourage you to engage with CSS folks on the subject to have that happen. You might consider proposing a TPAC session on this topic?
We do think that the name is a little misleading here as the "copy" part of the operation isn't the most important aspect; it's the conversion or transform that is the important action. Is this something that warrants a new API?
copyTolooks to be relatively new, is it possible to choose a better name?Finally, from a process perspective, we'd strongly encourage you to take this work to the Media Working Group repo rather than sitting in someone's private GitHub account.
Either way, this work looks like a useful contribution, so we're closing this as
satisfied with concerns.
No objections and posted
OpenedMay 1, 2024
こんにちは TAG-さん!
I'm requesting a TAG review of Conversion to RGB in VideoFrame.copyTo().
VideoFrame.copyTo()can convert pixel data to RGB pixel formatConverting YUV video frames to RGB is often required for processing them in libraries like TensorFlow.js and OpenCV.js. Previously the only possible way to achieve this was rendering the frame on a canvas.
Specifying
VideoFrameCopyToOptions.formatandVideoFrameCopyToOptions.colorSpacemake it possible to asynchronously convert frames to RGB pixel format by callingVideoFrame.copyTo()without having to use an extra canvas.Explainer¹ (minimally containing user needs and example code): https://gist.github.com/Djuffin/9e2f98025ead49998524510cfeed8d33
Specification URL: https://www.w3.org/TR/webcodecs/#dom-videoframecopytooptions-format
Tests: https://github.com/web-platform-tests/wpt/blob/master/webcodecs/videoFrame-copyTo-rgb.any.js
User research: N/A
Security and Privacy self-review²: https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c#file-webcodecs_contenthint_security_and_privacy-md
Primary contacts (and their relationship to the specification):
Organization(s)/project(s) driving the specification: Chromium
Key pieces of existing multi-stakeholder (e.g. developers, implementers, civil society) support, review or discussion of this specification:
External status/issue trackers for this specification (publicly visible, e.g. Chrome Status):
Further details: