#288: OffscreenCanvas new commit() and DedicatedWorker.requestAnimationFrame

Visit on Github.

Opened Jun 1, 2018

Hello TAG!

This is a follow up from: https://github.com/w3ctag/design-reviews/issues/141

I'm requesting a TAG review of:

This change addresses concerns pointed out at https://github.com/w3ctag/design-reviews/issues/141, namely:

  • the previous Promise commit() solution doesn't exist anymore, instead requestAnimationFrame has been added to DedicatedWorkerGlobalScope, and a render model has been described on it. This is semantic similar to what happens on Window (although not attached to a Document animation frame).
  • There's still a need for a commit() function, to support the new WebAssembly cases of a busy loop execution (that doesn't finish the context task) but that still wants to commit a render frame. This new commit() is a blocking function that returns as soon as the UA is ready to receive a new rendering of the OffscreenCanvas.
  • Apart from the request above, and since this is relevant to OffscreenCanvas, there's still a minor follow up PR on whatwg that enables (https://github.com/whatwg/html/pull/3708)[text rendering on OffscreenCanvas].

We'd prefer the TAG provide feedback as leave review feedback as a comment in this issue and notifying myself (@fserb) if needed.

Discussions

2018-07-10

Minutes

TL: The latest proposal looks like they have integrated a lot of our feedback. The explainer shows many things that were discussed at TPAC 2 years ago. Progress seems to be good. There is this thing called an Offscreen Canvas which is a proxy object that routes to another element in an actual document. It allows offscreen control to a dedicated worker. From there you can issue drawing commands to the context within the worker. You were expected to create an ImageBitmap, which is an opaque object.

KC: Is this like ImageData?

TL: No. This is different. It's sort of a blob. What has changed is that 1) they have figured out how to sync better, and 2) more proactively push frames from a dedicated worker to the actual document without jumping through too many hoops. There is also a PR which seemed a bit incomplete.

KC: Why would this not use SharedArrayBuffer? I assume it is because this is a specific case? Seems costly with copying.

TL: That's the point of ImageBitmap. It's a pointer to video memory.

KC: Is it possible to implement double buffering?

TL: Probably. You can create a bunch of these, although there is the risk of backpressure. There is an explainer, which isn't quite a spec.

KC: This is driven by WebXR?

DKA: Should note that there are no privacy considerations in the explainer.

TL: Does anyone know who raised this?

SM: What would be the actual privacy concern surface?

DKA: Well, there were specific measures from the TOR browser around <canvas>. (https://en.wikipedia.org/wiki/Canvas_fingerprinting)

Hadley: +1. It would be good to make the situation explicit in the explainer.

KC: Since this ties very closely into the GPU it would be possible to do timing attacks.

TL: The new proposal discusses batching the blit operations from the OSC to the placeholder <canvas> with commit. Second use case is for multiple views of the 3D scene, which requires the scene state to be in the same worker. The third is for WebVR. And so forth. The explainer could use a bit more detail, along with the PR. The rAF timing should be synchronized with the original context. (Scribe brainfart. Might have missed some details.) rAF in the worker is global, which is tied to the main window where the actual display canvas is.

KC: There is a possibility of rate changes with GPU changes or VR contexts.

TL: In the future the spec might accomodate this. One might use a WebVR rAF which runs at a higher framerate. The main purpose of this would be to run game engines built with WebASM from a worker.

KC: So commit() is blit.

TL: Yes. It's a method on a offscreen canvas, and it is a blocking operation. (previously it was not)

KC: Why is that not an option? (as in, a parameter)

TL: I believe this is because rAF serves a different purpose.

KC: So commit() is not something you'd always use.

TL: This seems specific to cases where you need to do blocking blit calls. Now that the background has been convered, the feedback is: The spec would currently only allow dedicated worker - since the other types don't have a strong binding with a specific window. The PR seems to suggest dedicated workers, but seems to be implicit about this. Will raise this. Curious about composition - are there cases where you want to draw parts in a worker and the rest in the main window. Not sure if this is a common use case that needs to be covered - should be worth bringing up. It's unclear what would happen when you call commit when you are in a rAF loop. commit seems to overlap the use cases that ImageBitmap covers. Code samples would be useful. Couple issues on the PR - text that has changed from async to sync, mentions about the user agents deciding specifics. Would like to see more work on this before it ships

2018-08-21

Minutes

AR: I'm increasing uncomfortable with the commit method...

TL: They're subbing out the old commit method for a new (blocking) one...

AR: Would having them on a call help?

TL: They've responded to some previous comments of which I haven't read the responses yet. Would like to look those over... Promppose we come back to this next week?

2018-12-19

Minutes

Travis: porting games over to web workers - e.g. through web assembly.

David: worklets are designed for code that's happening on the UI thread or some other existing thread - isolation but not threading...

Travis: holding pattern -

2019-01-22

Minutes

Travis: I might ping Ken Russell to see if they have made progress?