As it looks like WebCodecs is going through some design changes, we'd like to hold off on providing feedback until that part has been ossified.

The EncodedAudioChunk/EncodedVideoChunk parts of the WebCodecs spec aren't in flux -- which are the only parts this depends on -- so this should be safe to look at.

Good to know! Thanks. Will take a look.

@ylafon and I discussed this today.

We think the use-cases and the proposed API design is fine. There seems to be a bit of unanswered questions in terms of implementation details (e.g. MSE is hardware decoded, WebCodecs should also be hardware decoded, what if you don't have enough decoder capacity?) and whether or not this is a proper solution to the use-case at hand. However, we'll leave that discussion up to the group.

For now, we're happy to see this move forward. Thanks for bringing this to our attention!

@cynthia Glad to hear! Can you elaborate on your questions though? I don't think those questions are relevant to MSE. MSE makes no guarantees about whether media is hardware or software decoded or even anything about how the underlying decoder configurations are managed (intentionally so to allow user agent management of such resources).

The proposed mechanism is only about allowing authors to append already demuxed data into MSE, thereby skipping the demuxing which normally happens implicitly through the byte based append methods. So it's a strict cost reduction mechanism with otherwise no impacts on how MSE functions.

Did y'all have any feedback on whether these new append methods should be added as promise based? Should the interface use the non-promise style (updating, updateend events) for consistency until byte-based promise methods are available? Should we only add the promise based method for chunks once we have a promise based method for bytes?

Raising these questions again here, as we haven't yet received a response about them:

Did y'all have any feedback on whether these new append methods should be added as promise based? Should the interface use the non-promise style (updating, updateend events) for consistency until byte-based promise methods are available? Should we only add the promise based method for chunks once we have a promise based method for bytes?

Apologies for the delay.

Did y'all have any feedback on whether these new append methods should be added as promise based? Should the interface use the non-promise style (updating, updateend events) for consistency until byte-based promise methods are available? Should we only add the promise based method for chunks once we have a promise based method for bytes?

@plinss, @atanassov and I discussed during our VF2F today. And one of the concerns we had were if promises would provide the latency guarantees that the downstream application needs (given that current MSE is event driven, it should be okay?), and if a domain specific application of promises would make it painful to upgrade when byte-based promises are available. We don't have a strong preference for either, but we'd like to hear your thoughts on this.

Does the current design propose a generic path forward for a byte-based promise API? If so, we think promises are probably the right answer.

Did y'all have any feedback on whether these new append methods should be added as promise based? Should the interface use the non-promise style (updating, updateend events) for consistency until byte-based promise methods are available? Should we only add the promise based method for chunks once we have a promise based method for bytes?

@plinss, @atanassov and I discussed during our VF2F today. And one of the concerns we had were if promises would provide the latency guarantees that the downstream application needs (given that current MSE is event driven, it should be okay?),

Since the current MSE is event driven, I don't expect worsened latency due to apps instead awaiting Promise resolution/rejection via this new promise-based method for appending chunks versus what that latency would have been had it been event driven chunk-appending instead. In passing, I've heard that at least some MSE API users have already wrapped the existing event-driven byte-based append operations inside promises in their applications. Note also that the ergonomics of a promise-based API are simpler, too: the extra enqueueing of 'updatestart', and 'update' events are not done (nor 'abort' or 'error' in those rejection paths). It is unclear if existing API users depend on those events, or if their absence is negligible.

and if a domain specific application of promises would make it painful to upgrade when byte-based promises are available. We don't have a strong preference for either, but we'd like to hear your thoughts on this.

I'm not sure I understand this portion of your response. If and when Promisified versions of MSE appendBuffer() is available, it's unlikely that the event-driven versions would be deprecated. Hence, upgrading from event-driven to promisified byte-based append would not necessarily require urgent prioritization by the MSE API user, though (apart from the lack of probably superfluous extra events) there would not be a big change of actual functionality made available by the Promisified versions of those methods. I feel like I'm missing your point though.

Does the current design propose a generic path forward for a byte-based promise API? If so, we think promises are probably the right answer.

The path taken in MSE-for-WebCodecs' appendEncodedChunks() lays out a pattern that indeed could be followed by a Promisified version of the existing event-driven byte-based appendBuffer(): the draft specification PR (https://github.com/w3c/media-source/pull/302) for the Promise portion of the new appendEncodedChunks() reuses much of the existing state machine structure: at most one asynchronous operation can be in-flight at a time for a particular SourceBuffer, with well-defined start/complete/error/abort paths for the existing event-driven async operation (and now also for the Promisified appendEncodedChunks() async operation.

Thank you for the response. To be upfront, from a review point-of-view this is complete - and we are happy to see this proposal move forward. Thank you for bringing this to us and apologies for the process delay.

Hence, upgrading from event-driven to promisified byte-based append would not necessarily require urgent prioritization by the MSE API user, though (apart from the lack of probably superfluous extra events) there would not be a big change of actual functionality made available by the Promisified versions of those methods. I feel like I'm missing your point though.

The question/concern here was whether or not the interoperability with generic byte-base promises (when it happens) can be a potential future pain point if this is too domain-specific, and the question was about whether or not there ideas have been exchanged around that. I think the response addresses some of the questions we had, although I'll have to check with the other reviewers to be sure.