#285: Carriage of Web Resource in ISOBMFF

Visit on Github.

Opened May 24, 2018

Hello TAG！

I'm requesting a TAG review of:

  • Name: Carriage of Web Resource in ISOBMFF / Advance signaling of MPEG media and containers in web communication environments
  • Specification URL: N/A
  • Explainer, Requirements Doc, or Example code: See email
  • Tests: N/A
  • Primary contacts: @chrisn

Further details (optional):

You should also know that these documents were submitted to W3C under liaison from MPEG, for discussion and feedback. The documents themselves are not public, but the authors have confirmed that they are happy for TAG to provide public feedback.

We'd prefer the TAG provide feedback as (please select one):

  • open issues in our Github repo for each point of feedback
  • open a single issue in our Github repo for the entire review
  • leave review feedback as a comment in this issue and @-notify @chrisn

Discussions

Comment by @dbaron Jun 12, 2018 (See Github)

The email mentioned is apparently https://lists.w3.org/Archives/Member/tag/2018May/0004.html

Comment by @torgo Jun 12, 2018 (See Github)

Discussed on tag call today - question came up of why existing / in progress packaging format cannot be used.

Comment by @cynthia Jul 25, 2018 (See Github)

@chrisn how did the discussion with MPEG go on setting up a call? We'll put this on external feedback for now.

Comment by @chrisn Jul 25, 2018 (See Github)

The relevant people have been away at MPEG and 3GPP meetings in recent weeks, I'm following up with them right now.

Comment by @chrisn Aug 20, 2018 (See Github)

Is either August 28th or September 11th good for a call?

Discussed Aug 21, 2018 (See Github)

SM: These folks asked if they could join our call either 28th or 11th of Sept?

PL: Either date is OK with me.

SM: The esteemed chair can decide!

Comment by @cynthia Sep 4, 2018 (See Github)

Takeaway from last meeting: https://github.com/w3ctag/meetings/blob/gh-pages/2018/telcons/08-28-minutes.md

Summarized, the use cases seem valid. We have some general concerns around the security model (which has been improved, awaiting access to the updated spec) - particularly on whether or not this context should have access to "powerful features" which are only accessible in secure context, and whether or not this is a secure context, what the origin is, which we would love to have clarified in the updated spec.

Additionally, putting HTTP/packaging equivalent in the scope of a media container as with the current design looks like it could have some future scalability/flexibility issues - we'd love to see if a singular brand that specifies a web package, which can then be run would be an alternate option forward.

I missed the "Advance signaling of MPEG media and containers in web communication environments" in the review request, will look at that once we have access to the documents. Please let us know when the updated documents have been made available, and we will revisit this review.

Comment by @cynthia Oct 31, 2018 (See Github)

I have made attempts to figure out what the advance signaling proposal is trying to do or solve, but a simple explainer could be really helpful here.

Comment by @cynthia Oct 31, 2018 (See Github)

As for delivering web content through a MPEG container, we have looked at the proposal and believe that a more adequate approach would be to use web packaging to contain the content, which leaves room for extensibility and doesn't require reinventing a new mechanism.

On top of that, using signed exchanges this can prevent tampering with the content, which could be a extremely useful feature given that a DTV transport stream cannot be considered secure against MITM attacks. This is slightly harder when the package is not self-contained, and we believe that this would be more future proof than the proposed approach.

Comment by @dwsinger Oct 31, 2018 (See Github)

Hi Cynthia

the point of using a timed container is to get timed actions, which a general package doesn't offer. I'm not sure I understand why an MP4 file is intrinsically insecure -- it's as secure as the channel it's delivered over. Can you explain?

Comment by @cynthia Oct 31, 2018 (See Github)

@dwsinger the concern about tampering is because that DTV streams (the kind that goes through a coaxial cable) which aren’t secure.

Comment by @cconcolato Oct 31, 2018 (See Github)

The "Advance Signaling" document is not a proposal. It's a survey of the current techniques to do that and of the problems that people are confronted with. The solution may be W3C's Media Capabilities API, but it's not clear at this point.

Comment by @torgo Feb 5, 2019 (See Github)

@dwsinger we discussed briefly at f2f and agreed there is something to chat about here. Let's discuss how to be most effective.

Discussed Mar 12, 2019 (See Github)

DS: I'm chair of the group but didn't write the spec.

DS: 2 problems they are trying to solve in this spec - (1) you want to present a lecutre and [allow the user to manipulate the stream] as it is being watched. richer multimedia experience. also ad insertion. that's sync of web & media technologies (2) they also want to package that into a single [delivery]. that is where ISOBMFF comes in. if you are retreiving a package from a web server then local names can be treated as relative resources. there is a packaging desire and a sync desire. i think there is too much in the specification. they package many kinds of resources together - html, css, js, images, timed resources, overlays... This has been going through MPEG for a while. I think we need to get [various experts] and tech suppliers together in a workshop.

Yves: when the mpeg container is transfered through eme does that mean that eme black box can run js trackers and privacy invasive things without being noticed? is that desirable?

DS: The ability of the web page connected to the same origin - there is no attempt to package things from other origins or resources.

Yves: if it's encrypted through eme then the browser may not have complete control of what is going on.

Dan: concerns from TAG in context of this, and of HBBTV, have to do with security, e.g., knowing what the origin is, and that the security model of the web is so tightly bound to origin. Are these packages being retrieved over the internet, are they being broadcast, etc.?

DS: In order not to run afoul of multiple origin problems... where the spec says URL, it should have said local name, which you can compose with the package's URL. You got the package from somewhere, that's your base for any operation.

Dan: what if the package comes down over a broadcast channel?

DS: Like FLUTE, MBMS or ...?

Dan: or ...

DS: FLUTE names resources by URLs -- so you get a URL for that.

Dan: We also spent a lot of time on securing the web -- securing the web is about tying the security model to the HTTPS delivery as well.

DS: The only aspect I've looked at is the HTTP delivery. I don't think MBMS and FLUTE have enough security inspection because nobody's thought of deplopying them. You could construct an MBMS multicast that claims to have URL names claiming to come from example.com, and then presumably example.com would be same origin. That's not a problem with this packaging, it's a problem with FLUTE MBMS protocol.

Dan: Sounds like something to discuss at this theoretical workshop. Seems like packaging itself should be something more like signed exchanges.

DS: This problem would come up with simple HTML pages over FLUTE MBMS.

Dan: Then it's inherently unsound.

DS: We're talking about just synchronization of web resources and packaging.

Dan: We've been looking at signed exchanges -- if we say web packaging must have those kinds of properties, that seems to me to be something that could hlep -- we're removing the issue from the delivery.

Tess: From 10000 feet, feels like there have been so many attempts at packaging web content. Curently there are several simultaneous mutually incompatible attepts. Seems like publishing people, AMP people, MPEG people -- not talking to each other, may have incompatible requirements, will something work for all or most of them.

Dan: +1

DS: may also be an opportunity -- publishing people talk about books, audio books, narrated books. Video people approaching same user space from opposite ends. Image file format packaging untimed resources together -- new adif image format, in Mac OS, Windows, android builds. A lot of the technology already there. I have no particular axe to grind here. Getting the interested parties together to work out uses and what to solve.

Peter: curious where this intersects with ttml

DS: i've wanted to use a text track as a metadata track. ... one waay to tie slides to video time ranges...

Yves: in webrtc they have separated data, audio and vidoe channels there may be something here.

DS: w3c is obvious place to catalyze such a worksho.

Tess: there could be interest this year. typically w3c workshops are looking longer term and are about breaking new ground.

Peter: anything else TAG can do?

Tess: we had a finding on packaging?

Dan: Could we get this MPEG group looking at the security & privacy self check?

Peter: thanks david!

Dan: thanks!

--

Manu: the v.c. working group chartered to work on a data model - we were not cleared to work on a protocol or crypto - spec is going into candidate rec. Broad consensus. We've had multiple reviews from i18n. There is a group called "rebooting the web of trust" - meets every 6 months - through that venue we've had a lot of engagements out of the traditional community... so quite a bit of review.

We are looking for a sanity check from the TAG from an architectural perspective and on the data model itself. Multiple organisations implementing the spec. Being used in product now. Most of the focus after in CR - and if the work gets to rec is going to be standardizing moving the credentials around. Protocols...

Dan: what is the time frame?

DanB: asap

Lukasz: fairly large privact & security section - i like it. might even be too verbose.

DanB: i wanted to comment on lengh and verbosity - PING wanted our best thinking of how it cold be used by others. so we definitely look at your suggestions for compressing the text but PING has requested detail.

Dan: I will ask Hadley to feed back to you asap

Discussed Apr 3, 2019 (See Github)

Dan: I'll (hopefully) discuss with next week during the AC meeting.

Comment by @cynthia Sep 12, 2019 (See Github)

Sorry we've dropped the ball on this, @hober and I discussed this during our Tokyo F2F and were curious if any discussion or architectural changes have happened since the last update. (We haven't heard much on this, and since the spec isn't open hard for us to know)

We'd be happy to discuss a path forward on this next week if you folks are available, but the current design seems like it could be iterated on.

Comment by @cynthia Sep 12, 2019 (See Github)

(Figured I should explicitly mention @cconcolato and @dwsinger)

Comment by @chrisn Sep 12, 2019 (See Github)

Thanks, @cynthia. I'll let Cyril and David comment on current status of the MPEG draft standard, but the M&E IG is happy to continue conversation in W3C. Be good to discuss during next week if we can.

Discussed Feb 10, 2020 (See Github)

Sangwhan: I had a chat with Cyril (?) abot this at TPAC ... seemed unconvinced about the issues we raised. We can't tell them it's a bad idea, if they pursue a bad idea in some other standardisation body we can't really do antyhing about it.

... Suggest we leave a last comment saying we had a short chat about this... don't know why a bunch of new types, seems like it's trying to reinvent a file system... maybe this should be consiolidated into a package inside a video.

... Could end up creating a loop since you can put a video inside the data inside the video...

... I want to close this, I don't think we're going to get any traction here. Seems like a really bad idea

Comment by @ylafon Mar 5, 2020 (See Github)

@chrisn @cconcolato any news on this, especially if the specification changed since last time we reviewed it. Thanks,

Comment by @cconcolato Mar 5, 2020 (See Github)

The ISO standard has been published late December 2019: https://www.iso.org/standard/75492.html I don't know what version you reviewed last, but an important change done just before publication was the removal of the JavaScript tracks.

Comment by @chrisn Mar 5, 2020 (See Github)

We haven't discussed this further at W3C, although the team are exploring running a workshop on bundling of interactive media content. We should review the new standard in light of your previous feedback.

Comment by @hober May 28, 2020 (See Github)

Hi,

@cynthia and I took another look at this during the TAG F2F this week, and we think we can close this review. We encourage all of the interested parties to attend the upcoming related W3C workshop, which we hope will be an excellent opportunity for media people and web people to collaborate in this area.