#389: WebTransport
Discussions
Discussed
Jul 1, 2019 (See Github)
David: I started...
... to look at this this afternoon. It's pretty substantive. They're working on two specs, one defining protocol in IETF and one defining web APIs in WICG. And an explainer and three email threads. The first chunk of my comment was noting that the explainer felt a little short and asking if they could go deeper and explain how the parts fit together.
... Seems like this is still somewhat in flux, e.g. integration with Streams. In flux, "early review" status.
... Did yves get a chance to look?
Yves: I haven't had a chance.
Dan: Should I bump it? Organise a breakout?
David: Yes a breakout would be good. I was supposed to do that last week and I failed. It's across three continents.
Dan: I'll bump this to the 17th.
Dan: Good luck organising a breakout! l o l
Discussed
Jul 1, 2019 (See Github)
David: Sangwhan wanted to do breakout 2 weeks from now, so bump to 3 weeks from now?
Dan: Looks like there's been response to your initial comment.
Discussed
Jul 1, 2019 (See Github)
David: was supposed to schedule a breakout for this... for next week.
Peter: OK, bump this 2 weeks.
Comment by @dbaron Jul 3, 2019 (See Github)
A few very preliminary comments from spending an hour or two starting to look over the various documents:
From looking at the explainer, it seems like the "Detailed design discussion" and "Alternative designs considered" sections could perhaps be fleshed out a little bit more. One example of something I'd have liked to learn from those sections that I didn't would be how this proposal differs in scope from the previous RtcQuicTransport
proposal; I think there are probably a bunch of other big picture things that are in your head, aren't obvious, and would be useful for the TAG to understand while reviewing.
It would likely also be useful for the explainer to explain how this spec relates to the WHATWG Streams specification.
It would also be useful to understand why the spec currently doesn't expose these in workers, and whether there's a future plan to do so.
Is there a section somewhere, either in the explainer or one of the two specs, that explains how the two specs (the one in WICG and the one in IETF) fit together?
A few very detailed comments on bits that I noticed while jumping around to look at some examples of things:
I'd note that the if (!!chunk)
in one of the examples is a C++-ism and the !!
is not idiomatic in JS (although not really needed in C++ either).
receiveDatagrams() may only be called once at a time. If a promised returned from a previous call is still unresolved, the user agent MUST return a new promise rejected with an InvalidStateError.
This seems a little bit odd, and likely to be a bit of a programming hazard (producing rejections in unexpected ways, depending on timing/races). Would it make more sense to return an empty sequence? That said... in the context of understanding why this method is asynchronous -- would it even be guaranteed to be empty? If it's asynchronous because the datagrams are on their way over from another thread or process... might asking for more not yield more?
Also, it seems like the type of receiveDatagrams
is slightly wrong, since the prose says that it allows nulls in the result. It seems like it should be Promise<Sequence<Uint8Array?>>
.
Comment by @dbaron Jul 3, 2019 (See Github)
(I realize those detailed comments may quickly become irrelevant if there's a big refactor to integrate with the streams spec, as one of the other links suggested.)
Comment by @pthatcherg Jul 12, 2019 (See Github)
Responses to dbaron's comments from 10 days ago:
RTCQuicTransport is the p2p impl of a WebTransport. I made https://github.com/WICG/web-transport/issues/41 to track making that more clear.
The explainer has now been updated to use WHATWG streams. We plan to update the spec to use them as well after some more discussion on how WHATWG streams should work (in particular for creating send streams and bidirectional streams).
We definitely want WebTransports to work in WebWorkers, we simply didn't think to say that explicitly.
The W3C spec references the IETF specs. Victor made https://github.com/WICG/web-transport/issues/37 to track that.
I can't find any example of "!!". Perhaps that went away when we moved to WHATWG streams in the explainer.
On the nullability of receiveDatagrams(): you're right, that's a mistake. But we decided on a different approach that doesn't return null (see https://github.com/w3c/webrtc-quic/issues/124). We just need a PR for it.
On the goofiness of receiveDatagrams() only being called once at a time: that will probably go away with the switch to WHATWG streams.
Discussed
Aug 1, 2019 (See Github)
David: I should try to schedule that breakout - but it's been blocked by vacations.
[bumped
Comment by @dbaron Sep 10, 2019 (See Github)
@cynthia and I are currently looking at this in a breakout.
So far I'd note that we found that there are additional documents that seem critical to understanding this, in particular:
Comment by @cynthia Sep 10, 2019 (See Github)
Blunt and clueless question - what makes an Http3Transport unique over a QuicTransport (aside from the protocol being on another application protocol), and what service does this provide that fetch() is unable to?
We have a pile of other questions on this and the other two related specs - I can block out some time to get a full overview if you folks have time.
Comment by @pthatcherg Sep 10, 2019 (See Github)
Http3Transport allows at least 2 things over QuicTransport:
-
Going through HTTP proxies or firewalls that only allow HTTP.
-
Sharing a connection with HTTP traffic (to the same server with the same port rather than a different IP and port, and contained in the same congestion control context).
(Maybe Victor can remember more)
Vs fetch:
-
The server can push streams down to the client
-
The client and server can both send datagrams (unreliable, unordered).
Looking forward to a pile of questions :).
Comment by @pthatcherg Sep 10, 2019 (See Github)
By the way, the spec has been updated to use WHATWG streams everywhere (to resolve a previous question).
Comment by @vasilvv Sep 10, 2019 (See Github)
Http3Transport is currently the less thought-out part of the spec, as we are trying to better understand the use cases of people who want to use it. Its basic idea is along the lines of RFC 8441, in that it tries to provide a way to establish a bidirectional non-HTTP data channel within an existing HTTP connection. One of its main distinguishing features is that when the connection already exists, it allows to bypass a lot of overhead related to setting up a connection.
Both Peter and I are going to be in Google Tokyo on Friday. Everyone should also be at TPAC, especially the Wed-Fri half of it.
Comment by @dbaron Sep 12, 2019 (See Github)
@torgo and I are looking at this in a (second) breakout at the Tokyo face-to-face meeting.
One of the things we're talking about is the relationship between a bunch of different pieces of work that are ongoing.
If my understanding of what's going on is correct, it seems like WebSocketStream (see #394) is about designing a better (uses promises and streams, supports backpressure naturally) JavaScript API for the existing WebSocket protocol, and this work here is about designing a new network protocol (on top of a more modern substrate) and having a JavaScript API to go along with it. One question in our minds is what the path for developers to use the various things is; it seems like in so far as WebSocketStream and WebTransport's JavaScript APIs are doing the same things, they ought to be able to work the same way, so that it's as easy as possible for developers to migrate from WebSocketStream (which we understand to be a shorter-term project) to WebTransport. So, if this understanding is correct, we'd strongly encourage these JavaScript APIs to be developed in tandem.
That said, I'm not particularly confident of that understanding gleaned from across a bunch of different explainers and other documents, so we'd be interested to hear how it's wrong.
Comment by @pthatcherg Sep 13, 2019 (See Github)
We can discuss WebSocketStream vs. WebTransport more in person (higher bandwidth), but I did want to mention one thing that's fundamentally different: WebTransport has streams and datagrams (ReadableStream of bytes and ReadableStream of Uint8Array). WebSocketStream has messages (ReadableStream of Uint8Array). That makes WebSocketStream's WebSocketConnection look a lot like a WebTransport's DatagramTransport, except that for WebSockets, they are in order and reliable and for DatagramTransport, they are out of order and unreliable.
Comment by @cynthia Sep 16, 2019 (See Github)
I'm interested in setting aside some time to discuss these details. Are you meeting during TPAC separately or is this is happening as part of WebRTC?
Comment by @vasilvv Sep 16, 2019 (See Github)
We have a breakout session on Wednesday: https://w3c.github.io/tpac-breakouts/sessions.html#webtransport
We also intend to meet during the WICG meeting, though I believe the WICG agenda has not been finalized yet.
Comment by @dbaron Dec 3, 2019 (See Github)
It looks like we missed our opportunities to talk about this in person in Tokyo or at TPAC. @ylafon and I are cycling back to this in a breakout at the TAG's Cupertino meeting.
I'm curious if there's something written down that explains the relationships between these various specs at a high level, related to the brief discussion that started in https://github.com/w3ctag/design-reviews/issues/389#issuecomment-530694381. (It's also useful for understanding how and why the Javascript APIs for them should differ.)
I'm also curious if there's something that explains the handling of consent to communicate (along the lines of the discussion that started in https://github.com/w3ctag/design-reviews/issues/296#issuecomment-418572658 and continued a bit in #303 and #304). Is there an intent that Http3Transport
and QuicTransport
would differ substantially (e.g., with the former using CORS and the latter using ALPN as described in 17.3 Protocol Security?). That said, now that we're discussing it, we're not even sure we understand the design around QUIC and HTTP3 and what the relationship is (although I think my understanding is that HTTP3 is built on top of QUIC and other things could use QUIC directly... but I'm not sure exactly what they do and don't get if they do that).
Comment by @dbaron Dec 4, 2019 (See Github)
For what it's worth -- maybe a more general way to frame my previous comment is that perhaps it would be helpful if the explainer assumed a bit less knowledge about the space. It doesn't need to explain the web from first principles, but having a few more paragraphs to explain the basics of how this fits in to the things it builds on or connects to would probably be helpful.
Discussed
Jan 1, 2020 (See Github)
Kenneth: Looks like no update since December...
Yves: Could use the constructor to scan hosts and ports. Might want to return (?) an object in a promise so it can check if it's allowed or not...
Kenneth: Did you leave that feedback somewhere? I just added the Pending External Feedback label.
Yves: I will add that.
Kenneth: David asked about handling consent to communicate in his comment. Might need to bring someone on to a call...
Yves: Perhaps we could ask David to invite someone from Mozilla...
Kenneth: Seems to be a pretty fundamental technology as well, definitely want the right people looking at this.
Yves: We need more expert review on this.
Alice: How...
Yves: Would be good to have Dan weigh in on whether to use plenary time for this. I'll talk to Dan about how to organise that.
Kenneth: Only three examples in the spec... and they're actually very similar. Oh.. there's more in the explainer.
... APIs on the web don't really fit together... mediaSource, they have to wrap everything in promises.
Yves: There's an existing issue regarding violation of same-origin policy.
Comment by @plinss Jan 15, 2020 (See Github)
dbaron to invite Martin Thomson to a future call
Comment by @cynthia Mar 3, 2020 (See Github)
@ylafon and I discussed during our Wellington F2F.
We think this proposal is extremely interesting, and will bring a whole new set of capabilities to the web. However, we also consider this a pretty powerful feature - and was wondering about the implications of the constructor magically connecting, hence not being able to be a promise. (which would allow gating it behind a permission) We also had some concerns about port scanning through this mechanism (e.g. through timing based testing, albeit it would only work on UDP ports for now) but it does feel like gating this behind a permission would be good enough as a mitigation.
Comment by @cynthia Mar 3, 2020 (See Github)
@plinss @dbaron ping on the Martin Thomson call issue noted above.
Comment by @aboba Mar 4, 2020 (See Github)
@cynthia The WebTransport constructor was modeled after the WebSockets constructor which also immediately establishes a connection.
To address the port-scanning issue we could prevent WebTransport from connecting to ports on the "bad ports" list.
Comment by @cynthia Mar 5, 2020 (See Github)
@aboba WebSockets is less powerful, and hence was never gated. This is a significantly more powerful API, so I think different considerations has to apply. As for the bad port blacklisting, those I believe are TCP ports; not sure if that blacklist would apply here.
Comment by @vasilvv Mar 5, 2020 (See Github)
What are the specific ways in which WebTransport is more powerful than WebSocket that are of security concern here?
Comment by @annevk May 14, 2020 (See Github)
https://github.com/WICG/web-transport/issues/110 seems somewhat relevant for the TAG review. (Also applies to some of the other class names I realized, e.g., QuicTransport.)
At least from Mozilla's review of this I don't think we found reason to permission-gate this and even "bad ports" might not be needed due to the specifics of the networking handshake. @martinthomson might recall this better. Having said that, I believe @ricea still plans on doing a stream-based version of WebSocket
and it seems this should be more analogous to that than to WebSocket
itself.
Comment by @martinthomson May 14, 2020 (See Github)
Yes, I think that we can avoid the port list and the permissions. Port scanning can be addressed by having the connection fail with a minimum timeout and no error codes that expose how it failed (maybe with some exceptions that we can carefully vet). After that, you leak no information until the server consents to communicate with this origin.
The big concerns are that we keep this and the new WebSocket stuff in sync so that they share what they need to (and that we don't do the same work twice). I'm assured that there is good coordination with Google between the teams, so that much is good.
Comment by @aboba May 14, 2020 (See Github)
@martinthomson Are you referring to WebSocketStream? There are some differences, but they were closed as "won't fix": https://github.com/WICG/web-transport/issues/50 https://github.com/WICG/web-transport/issues/51
Comment by @annevk May 14, 2020 (See Github)
In light of https://docs.google.com/document/d/1La1ehXw76HP6n1uUeks-WJGFgAnpX2tCjKts7QFJ57Y/edit those make sense I think, https://github.com/WICG/web-transport/issues/54 appears unaddressed still.
Comment by @martinthomson May 15, 2020 (See Github)
Not sure that "Peter talked to Adam" is sufficient for a decision, though I appreciate that this is a personal submission, so it does have a different standard.
Now sure how the WebSocketStream API is necessarily distinct based on the proposal.
Comment by @ylafon May 26, 2020 (See Github)
Sangwhan and just discussed this and we are fine with not gating with a permission to avoid port scanning as long as there is no way to infer what is happening by doing time-based analysis (connecting to a filtered port takes far more time than a blocked one or one responding with another protocol).
Comment by @cynthia Sep 24, 2020 (See Github)
@dbaron and I discussed this during a breakout today, and concluded that given the above port scanning mitigations are in place we are happy to see this proposal move forward. Thanks for bringing this to our attention, and we plan to close this after discussing with the group.
We'd like the group to actively engage in review with relevant experts while development progresses. (In particular, networking is one of our weak points, so we might have missed something important.)
Discussed
May 1, 2021 (See Github)
Sanghwan: i set it to proposed close - i think we shoudl close it.
Dan: OK
Yves: yeah and re-open if they want more feedback.
[closed]
OpenedJun 21, 2019
おはようTAG!
I'm requesting a TAG review of:
Further details:
You should also know that...
We'd prefer the TAG provide feedback as (please select one):