#514: Client-side video editing (MediaBlob)

Visit on Github.

Opened May 19, 2020

Hello TAG!

I'm requesting a TAG review of Client-side video editing (MediaBlob)

MediaBlob is a high-level API that enables video editing on the browser. This API solves these common overheads/workarounds that web developers currently use for editing media:

  1. Use external JavaScript/WebAssembly libraries usually in several MBs (affects bandwidth and may not be efficient)
  2. Server roundtrip (affects both time and bandwidth)

This API is introducing trim, concatenation and split operations on the media files.

Further details:

  • I have reviewed the TAG's API Design Principles

  • The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG

  • The group where standardization of this work is intended to be done ("unknown" if not known): unknown

  • Major unresolved issues with or opposition to this design: None yet

We'd prefer the TAG provide feedback as:

🐛 open issues in our GitHub repo for each point of feedback

Discussions

2020-06-15

Minutes

Ken: there is overlap between this and web codecs. I have been following discussions on blink-dev - looking at this API, for me it seems nice. This seems like a good API. but it was pointed out on blink-dev that these are not the most common use cases. But a comment people have given is that it doesn't scale.

Tess: the lesson from that is layering. This captures common use cases and those should be easy. If it can be defined in terms of the lower level stuff then that's good. If the convenient stuff works for you then great.

Rossen: it does hit and address the most common cases which are ~80% of what people need. Where the strggle that comes out - it was one of the issues that was raised - one of the main weaknesses of the API is splicing differently encoded and different types of media... This API becomes lossy for that. It will have to renormalize to something that loses the original quality. On the flip side, from experiments that [MS] team has run with real media, they have run a number of partner use case verifications and the results were overwhelmingly positive. Today this is very painful. In the most common use cases - e.g. trimming a video, it helps.

Ken: would it be possible to show a slider with frames?

Rossen: scribing? I think this is capable through video element and API. the point is what happens next. Today you send it back to the server. There are some JS libraries that have performance cliffs. As a general principle, I'm aligned to what Tess said about reducing developer pain.

Ken: some examples missing for how to tie this together with playback. Seems like something very separate from playback. As it's supposed to be simple, some examples of how to tie it together with playback would be useful.

Rossen: yes, the main selling point is ergonomics. Showcasing an end-to-end authoring case would be good. I will add the comment. Push a couple weeks?

Ken: O

2020-07-13

Minutes

Ken: touches on the same area as webcodecs. They are looking about how to use this on top of web codec. Hard dependency. This API is a more user friendly API running on top of codecs.

Ken: propose close

Sangwhan: 2nded

[closed

2020-08-17

Minutes

Alice: I see this is propose close...

Ken: Yeah, until the propose something new with WebCodec as the base primitive. Now this is about providing convenience APIs, which they say they're discussing in another issue.

Yves: Could this be related to Insertable Streams..?

Ken: I don't know.

Yves: No requirements to have streams in real time... could stop, do some editing and continue. One of the goals of insertable streams was to allow things like adding backgrounds.

Ken: Seems anyway like this won't progress until there is this new thing...

Yves: Should we ask if they still need the review...?

Ken: Yeah, I'll write something.