#889: Allow transferring ArrayBuffer into WebCodecs object constructors
Discussions
2023-10-16
Dan: media wg.. because it has to do with codecs. They have listed firefox and safari as n/a.. not clear why from the minutes they linked to
Amy: they are asking a specific question they want guidence on
Dan: need to research more into web codecs..
Sangwhan: this is not about a proposal, this is asking us for an opinion on a particular pattern. A missing principle.
Dan: presumably they will need to deal with multistakeholder.. just found it weird to see n/a.. if they think it doesn't require consensus.. feel we should ask about that even if it's tangential
Sangwhan: disagree in this case. It's not a proposal yet. They want our guidance then go back to the WG, iterate, then the multistakeholder discussion will happen.
Dan: okay
Sangwhan: the problem is around trying to be consistent vs usable. have to think about this. Transferrable objects... we don't have a recommendation for how to approach these things. I believe they copied existing patterns and what they're asking is it doesn't quite work well for them. Related to concept of transferable objects - transfering objects across process boundaries...
Peter: saying they don't really need the full pattern but maybe it's okay?
Sangwhan: something for us to think about. The transferrable objects paradigm is designed around being able to transfer objects across process boundaries, underlying implementation wise. For web codecs you're not crossing process boundraies, just thread boundaries. Which has a completely different implication.
Dan: they talk about transferring objets that can only be accessed from one thread at a time.. are they really talking about transferring objects between processes?
Sangwhan: the initial concept was created for processes boundraies, created for workers. When you transfer an object from the main thread over to a worker that crosses an entire boundarie of a process, tricky to do lock/unlock. Execution context is on thw orker side. Unless you do something like shared memory to sync across two processes, which is a nightmare.. you want to just hand it over. What they're seeing here.. will have to check.. I think webcodecs is just a thread transfer, so you don't need the compelxities surrounded around transferrable objects because it's not as complicated.
Peter: that's an implementation detail right? Couldn't it be a process boundary at some point in the future in someimplementation?
Sangwhan: right.. coudl be.. but we don't have a pattern that distinguishes process and thread boundaries or provide the ergonomics that are necssary. Transferrables are not nice.
Peter: also think we shouldn't be exposing the fact you're crossing a thread or process boundary in a web api. Can explain this as nothing more complicated than... you don't need the data any more so we'll clear out your reference to it and you don't have to worry about copying the data. Whether it's process boundary or something else the consumer shouldn't have to care. Agree we don't have a good way fo doing that in a way.. this isn't a Rust API where we're handing ownership of this object.
Sangwhan: not something the language supports well. We're retrofitting this mechanism through api surface, it's sort of ugly
Peter: yeah. I see maybe inventing some wrapper object instead of the array buffer, make a transferrable object, give the array buffer to that, pass the object..
Sangwhan: right
Peter: I don't think adding an extra boolean saying we're transferring this as a separate flag is any better ergonomically than the array. The array is clumsy but the boolean is not extensible or obvious what's being transferred
Sangwhan: the boolean itself.. depends on the api might not be adequate
Peter: yeah. And if they ever extend the api ..
Sangwhan: if you have a constructor where you're talking an object initially, the boolean at the top level is it inside the dictionary that goes into the constructor..
Peter: never used this but in post message.. you're sending one object then an array of trnasferrable objects. Presume they could be things referred to by the original object? I could be passing an object that has 3 properties, two of them are in the transferrable list. Deep copy of the whole thing exceopt whatever is in the transferrable list is going to get moved instead of copied
Sangwhan: yeah. Array approach is also not nice
Peter: it's not, but it's what we have
Sangwhan: I think they're okay with following that pattern but I suppose they reached out to us because it smelled funny. When the ergonomics of an existing pattern is sort of off. I think it's something we should think about and possibly provide architectural guidence on how to improve this going forward. It's very hacky.
Peter: my thoughts. If their question is should we use a boolean or a transfer list, betweent hose, use a transfer list. But even better would be .. not their problem to solve, but us and/or a task force to come up with a better design for how to do this in the future.
Sangwhan: realistically us and TC-39 kind of problem
Peter: yeah
Sangwhan: you would want to encode.. ultimately move syntax into a language without pointers or memory managers
Peter: who is our current liason with tc39?
Sangwhan: Dan Erinberg(?)
Dan: I can raise this with Dan tomorrow
Sangwhan: I will also message him privately. I think this should be a larger group discussion. It is a problem. We and tc39 should come up with a problem statement and straw proposal. For the time being my recommendation would be to use transferrables for now. [will leave comment]
Peter: +1 we don't ask them to invent a better mechanism, but we should try to get one designed
Sangwhan: We definitely don't want a one off case for this
2024-03-11
Amy: they've asked a very specific question, maybe they know by now
Dan: remember a discussion with Sangwhan around this, no conclusion, didn't seem like architectural concern https://github.com/w3ctag/meetings/blob/gh-pages/2023/telcons/10-16-minutes.md#allow-transferring-arraybuffer-into-webcodecs-object-constructors
Amy: looking at S&P there are increased fingerprinting and exploitations on underlying codecs in their considerations sections, but maybe not significant enough to worry about - I don't know enough about codecs to know. There are mitigations mentioned.
Amy: the spec has editors from Google, Microsoft and Mozilla... so... [multi-stakeholder]
Matthew: a lot of info people could get out of the raw stream.
Amy: shipping
Matthew: ...since December
<blockquote> Hi @DjuffinMy apologies it's taken us so long to get back to you on this. I realize things may have moved on signifigantly since you filed this review. We did have a fairly detailed discussion on this review in October but unfortunately the output didn't make its way into this issue. Specifically you've asked "Should we just use a boolean value instead of a transfer list to indicate transferring of the ArrayBuffer contents to a WebCodecs object". In the absence of our feedback have you made any decision in the group thus-far on this? In any case, we don't see any issues and we're happy with the multi-stakeholder story here.
</blockquote>2024-03-18
Dan: I think we should close on this basis.
Matthew: yes.
Max: yes.
closed as satisfied
OpenedAug 30, 2023
こんにちは TAG-さん!
I'm requesting a TAG review of "Allow transferring ArrayBuffer into WebCodecs object constructors".
We're working on adding transferable buffers to WebCodecs object constructors. We've copied the transfer-list approach used by structuredClone and postMessage.
However a concern have been raised that this mechanism isn't ideal: Should we just use a boolean value instead of a transfer list to indicate transferring of the ArrayBuffer contents to a WebCodecs object. Arguments:
We've been asked to solicit TAG guidance on whether copying this mechanism is appropriate in terms of TAG principles around consistency.
Further details:
We'd prefer the TAG provide feedback as (please delete all but the desired option):
💬 leave review feedback as a comment in this issue and @-notify @Djuffin