#754: Fetch streaming upload

Visit on Github.

Opened Jul 5, 2022

Wotcher TAG!

I'm requesting a TAG review of fetch streaming upload.

[One paragraph summary of idea, ideally copy-pasted from Explainer introduction]

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • Relevant time constraints or deadlines: [2022-07-19]
  • The group where the work on this specification is currently being done:
  • The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue):
  • Major unresolved issues with or opposition to this specification:
  • This work is being funded by:

You should also know that...

[please tell us anything you think is relevant to this review]

We'd prefer the TAG provide feedback as (please delete all but the desired option):

🐛 open issues in our GitHub repo for each point of feedback

☂️ open a single issue in our GitHub repo for the entire review

💬 leave review feedback as a comment in this issue and @-notify [github usernames]


CAREFULLY READ AND DELETE CONTENT BELOW THIS LINE BEFORE SUBMITTING

Please preview the issue and check that the links work before submitting.

In particular, if anything links to a URL which requires authentication (e.g. Google document), please make sure anyone with the link can access the document. We would prefer fully public documents though, since we work in the open.

¹ We require an explainer to give the relevant context for the spec review, even if the spec has some background information. For background, see our explanation of how to write a good explainer. We recommend the explainer to be in Markdown.

² A Security and Privacy questionnaire helps us understand potential security and privacy issues and mitigations for your design, and can save us asking redundant questions. See https://www.w3.org/TR/security-privacy-questionnaire/.

Discussions

Comment by @torgo Jul 20, 2022 (See Github)

Ho @yutakahirano - can you please update your explainer with a bit more detail? Please have a look at the explainer explainer for some detail on this. A good explainer should start with user needs - rather than developer needs. There's a purpose to this - to make sure the technologies we are adding to the web are build to meet user needs or solve user-facing problems. Thanks!

Comment by @yutakahirano Jul 21, 2022 (See Github)

@torgo This allows web developers a way to upload unbounded streams. On the other hand, web developers can work around the current limitation, via multiple fetch requests or other APIs/protocols. There are other problems for such mitigations (more network resources, more latencies, extra developement cost for another protocol, etc.) but these are problems for developers, not for end-users. Hence it is difficult to state clear user benefits, although I believe the new feature ends up benefitting end-users in the end, via more efficiency and less developement cost.

Comment by @torgo Jul 21, 2022 (See Github)

Hi @yutakahirano that's great - can you please articulate that user benefit at the beginning of the explainer. Something like "More efficiency and lower development cost for streams means better, more performant, more reliable web applications when those web applications make use of streams."

Comment by @yutakahirano Jul 25, 2022 (See Github)

Thanks, I updated the explainer.

Discussed Aug 1, 2022 (See Github)

Max: noticed that this is the first version and it can only support half duplex mode - but they plan to support full duplex mode. In their explainer they explain that this is the first version... we discussed full duplex in WHATWG... Will leave comment.

Yves: to me it looks like a valuable addition - they are using streams which is the "right" way of doing it.

Comment by @maxpassion Aug 30, 2022 (See Github)

Hi @yutakahirano, we had a discussion in our TAG meeting today. In the explainer, you mentioned that the current version support only "half-duplex" and the future version may support "full-duplex", is there any mechanism that deals with the compatibility of different versions?

Discussed Sep 1, 2022 (See Github)

Max: I left comment and they replied - they only support half duplex mode - the full duplex mode mentioned in a note. When the full duplex streaming mode implemented... There should be more issues between compatability between different versions.

Tess: I think I found the update... if you look in the document for "caveats" look at the para before...

Tess: could be "currnetly web applications can't upload web data sets without knowing the data set up front..." I feel the request [for user needs] isn't an unreasonable one.

<blockquote> Hi @yutakahirano thanks for the updates. We'd like to see more detail on the user need and expected use cases for this. For example "Currently when users need to do xxx the situation is yyy. This creates problems because zzz. This technology solves these problems by qqq." This should be at the top of the document explaining what the new technology proposal is. We're also reviewing the API and we appreciate your feedback regarding the modes. Also thank you for going through the security & privacy questionnaire and providing answers. Can you bring this document into the same repo as the explainer please (rather than a google doc)? </blockquote>
Comment by @yutakahirano Sep 6, 2022 (See Github)

Sorry for the delay!

Currently "half" is the only valid value, "full" is just mentioned in a note. Hence currently new Request('/', {duplex: 'full'}) throws an exception (if the browser supports streaming upload). When full-duplex streaming upload is specified and a browser starts supporting it, the browser will accept new Request('/', {duplex: 'full'}).

Comment by @torgo Sep 29, 2022 (See Github)

Hi @yutakahirano thanks for the updates. We'd like to see more detail on the user need and expected use cases for this. For example "Currently when users need to do xxx the situation is yyy. This creates problems because zzz. This technology solves these problems by qqq." This should be at the top of the document explaining what the new technology proposal is. We're also reviewing the API and we appreciate your feedback regarding the modes. Also thank you for going through the security & privacy questionnaire and providing answers. Can you bring this document into the same repo as the explainer please (rather than a google doc)?

Comment by @yutakahirano Sep 30, 2022 (See Github)

Done: https://github.com/yutakahirano/fetch-with-streams/blob/master/streaming-upload-security-privacy-questionnaire.md

Discussed Oct 1, 2022 (See Github)

Max: We are expecting an update to the explainer .. more use cases... They did send a link to the answers to the questionnaire... You suggested to update their explainer...

dan leaves comment

Discussed Oct 1, 2022 (See Github)

Max: they responded - in the explainer they've added a section talking about the benefits. Some end user benefits.

Dan: I think the discussion is editorial. Do you have concerns about the api?

Max: I don't have other concerns

Dan: we could close this closes with comment

Comment by @yutakahirano Oct 1, 2022 (See Github)

cc: @jakearchibald

Comment by @torgo Oct 11, 2022 (See Github)

Hi @yutakahirano thanks for the updated questionnaire responses.. I think we're still looking for some more info in the explainer relating to use cases and user need (from the end-user PoV)?

Comment by @yutakahirano Oct 12, 2022 (See Github)

"Currently when users need to do xxx the situation is yyy. This creates problems because zzz. This technology solves these problems by qqq."

I think they are already explained in the doc.

The end-user benefit is described as:

These performance shortcomings and cost of additional logic will impact end-users. In other words, this feature will help end-users enjoy web applications with better performance, more reliability and less cost.

Sample use-cases are listed as:

  • Web developers can upload unbounded data to the server.
  • Web developers can implement streaming processing easily.

The kwown workarounds and their shortcomings are described at the Without the feature paragraph.

I like the current structure because xxx, yyy and zzz are long. If we had them in a single paragraph, it'd be very hard to understand the paragraph.

Comment by @torgo Oct 18, 2022 (See Github)

@yutakahirano Ok we're happy to close this on that basis. For reference: I think you do need to spend more time thinking about how to explain this feature and its benefits - especially when it gets documented in places like MDN where developers are not going to have the same context in their heads so may need a little more explanatory text. That's part of the point of asking for explainers, by the way.