#516: Blink Shipping Process

Visit on Github.

Opened May 28, 2020

Hello TAG!

I'm requesting a TAG review of the Blink Process for Launching Features to the Web.

Given how closely we, TAG, work with teams from the Blink and Chromium community it will be great to review their process and consider if there are ways we could work more efficiently together.

  • Explainer¹ (minimally containing user needs and example code): https://www.chromium.org/blink/launching-features
  • Primary contacts (and their relationship to the specification):
    • Alex Russel @slightlyoff, Google
    • Chris Wilson @cwilso, Google
    • Chris Harrelson @chrishtr, Google
  • Organization(s)/project(s) driving the specification: Google, Chromium
  • Key pieces of existing multi-stakeholder review or discussion of this specification: N/A
  • External status/issue trackers for this specification (publicly visible, e.g. Chrome Status): N/A

Further details:

  • I have reviewed the TAG's API Design Principles
  • Relevant time constraints or deadlines: none
  • The group where the work on this specification is currently being done:
  • The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue):
  • Major unresolved issues with or opposition to this specification:
  • This work is being funded by:

We'd prefer the TAG provide feedback as (please delete all but the desired option): 💬 leave review feedback as a comment in this issue

Discussions

Comment by @cwilso May 28, 2020 (See Github)

We're actually working on an update to the process, and it would be best to focus on that rather than what's on the page today.

Comment by @alice May 28, 2020 (See Github)

@cwilso Sure, should we wait? Or is there a draft you'd like us to look at?

Comment by @tomayac May 28, 2020 (See Github)

I'm requesting a TAG review of the Blink Process for Lunching Features to the Web.

@atanassov Rather hilarious typo :-) The TAG eats our launch process not for breakfast, but for lunch…

s/Lunching/Launching/

Discussed Jun 22, 2020 (See Github)

Dan: i think we should punt this to the plenary as well

Discussed Jul 13, 2020 (See Github)

[bumped

Comment by @atanassov Aug 14, 2020 (See Github)

@cwilso are we getting closer to review the new updates to the process?

Comment by @cwilso Aug 14, 2020 (See Github)

Yes! Sorry, got distracted and forgot to update this issue. We pushed the process live last week - the documentation at https://sites.google.com/a/chromium.org/dev/blink/launching-features has been updated. The "New Feature Incubation" section of this page is probably the most interesting/relevant here (the rest of the types of features are largely less controversial).

I'm happy to sit in on a live discussion if a Q&A would be helpful to the TAG, or answer questions asynchronously.

Discussed Aug 17, 2020 (See Github)

Alice: Let's bump this to next week when Dan is back.

Discussed Aug 17, 2020 (See Github)

Tess: We had a good discussion about it in Breakout A. One of the things was to step back and discuss the goals of why the Blink project would involve TAG and vise versa.

From Blink point the expectation is that:

  • TAG reviews should be useful.
  • TAG reviews shouldn't be a blocker

From TAG side the expectation is that:

  • TAG isn't percieved as a rubber stamping body to ship features
  • TAG having a positive review shouldn't be given the weight of "this has passed sufficient wide review to be a standard"

Further, timing of when TAG is engaged is important. We concluded that we weren't sure if our decision making is going down the right path so we thought we can use the help from someone much closer to the project. (Tess restating Breakout A discussoin around moving steps from 3 to 2).

Tess: Alice, what do you think?

Alice: My raw thoughts are... the framing of the expectations from both sides is really good. Having affordances for us, TAG, to be able and handle revies on time and with high quality is important. An explicit exit criteria out of each step can be improved. Step 3 is supposed to be an iteration step.

Tess: Yes, but at this point the expectation is to be feature complete.

Alice: Incubation depends on having code and being able to iterate with those who request the features. Another point that seems to be missing is calling out polyfills.

Peter: Yes, and polyfills can be used for incubation too.

Alice: The idealized theory is, you're writing code that is a proof of concept that you play with... but in reality this is usually what gets productized in the long run. By the time you have something to review, some level of buy-in is done. Similar to code review where you get a bunch of feedback after you write code and you need to change... compared to paired programming where the feedback is much more real time.

... My question would be - what kind of artifact could we review that doesn't have the problem of buy-in

Tess: Explainers are such an example which needs to be done much earlier before you write code.

[Discussion] about having early engagement that will be timely (say two week turnaround for example). Perhaps we can have a different repo ideation-reviews that will allow us to validate the use case, suggest prior art that could be missing and agree on rough path forward.

Discussed Aug 17, 2020 (See Github)

[all reading the proposal]

Tess: We are mentioned a number of times and it seems like we come into play after step 3, which is after feature-complete. That sounds a bit too late but the next section reads that this is when feature development starts...? ... It feels like it will be good to have an earlier input from TAG/Standards. Perhaps this isn't there so that we don't have as heavy of a load on reviews as we do now?

Peter: Not sure if WICG should be the gate keeper for standardization. Not looking for negative signal but rather a positive signal of sorts.

Tess: Right, and this is somewhat what you'll need to get going with WICG... some engagement and thumbs up from others on discource. ... Also, previously we used to get a number of reviews for documents on random personal repos, having the WICG step there should help.

Peter: TAG review at Step 3 probably is better done at Step 2

Tess: And move the "you should have TAG sign off by now" to Step 4; it should be part of "evaluating readiness to ship".

Rossen: [many things which sadly didn't get minuted]

Tess: From the Blink side, I think they want our review to (A) be useful, and (B) not delay their product schedule. From our side, we want to feel like (C) our reviews (can) be impactful, and (D) our reviews are not rubber-stamping. Moving starting TAG review from step 3 to step 2 could help with (C), and moving completion of the review from step 6 to step 4 may help with (D).

[failed to minute]

Peter: Note that they require TAG review, they don't require favorable TAG review.

Peter: Deprecation process should likely involve a standards body at some point.

[failed to minute]

Comment by @cwilso Aug 28, 2020 (See Github)

I presume from the milestones you've been internally discussing; maybe it would be useful to have an interactive discussion?

Comment by @plinss Aug 28, 2020 (See Github)

Yes, we plan on inviting you to a call in the near future.

Discussed Sep 14, 2020 (See Github)

Ken: some people have been using TAG review as "it's fine" - but our review is not exhaustive. Some of these don't get wide review in W3C. Should that be part of blink process as well? Horizontal review.

Rossen: in talking about this - is Chris going to join us at some point?

Peter: we wanted to get group consensus before inviting Chris.

Dan: isn't there a new wide review process?

Peter: part of process 2020?

Yves: W3C review is still per domain.

Ken: we could add it to their github template - asking or encouraging them to do wide rview...

Peter: we did discuss this here (8-17 minutes)

Alice: longer conversation in the plenary in that same doc.

[reviewing what we said]

Rossen: as a first step we can dump these things back into the issue and have a reply in the issue before moving to a synchronos discussion. I can summarize in the issue. We discussed afew. The things that resonated the most with me were : what we expect from the review and what we believe that the blink process expects from the review an then based on that what is the sequence / timing? One main point: perhaps to make the review process more streamlined / positive, to have the initial quck-signal or "idea review" - a "signal review" and have clear expectations on both sides - a time box wihtin the TAG should respond and if they don't hear they can assume it's fine. And then some minor points of feedback - about putting the TAG review into an earlier step in the process.

Rossen: any other major points

Comment by @atanassov Sep 23, 2020 (See Github)

We had a chance to review and discuss the Blink process during few of our breakouts. The following is a summary of our discussions.

In general we feel that the TAG review expectations from the Blink side can be summarized as: (A) Be useful (B) Not delay or block the Blink product schedules

From TAG's side, we want to feel like: (C) TAG reviews can be impactful (D) TAG reviews are not perceived as rubber-stamping

To that effect we think that moving the start of TAG reviews from step 3 to step 2 could help with (C), and moving completion of the review from step 6 to step 4 may help with (D).

We do want TAG reviews to be able to be completed in a timely manner and at high quality. Having an explicit exit criteria for each step of the Blink shipping process can help a lot. For example, step 3 is supposed to be an iteration step, right?

Another missing point from the process is calling out polyfills. More often, polyfills are used for incubation and that's when/where most technical discoveries and decisions are made. There is a missed chance for TAG to engage here and what's worse is that such features get easily adopted and become de facto standards/expectations.

Another observation is that most of TAG's engagement and feedback comes a bit too late in the process. In turn, the feedback is harder to accept since ideas are closer to solutions that are sometimes ready to adopt and ship. One idea that seems to have good traction with TAG is to introduce an early stage review (let's call it Ideas Review for now). This stage should be very lightweight and time-bound in terms of TAG review turnaround (say 2 weeks turnaround time or consider accepted). In order to meet such time expectations we would expect explainers to be short and easy to review. During idea-reviews we will have a chance to look for general platform consistency, best venues for said ideas as well as spot any prior-art lessons we can suggest to authors so they can get familiar with before moving to design.

@cwilso, happy to get together and discuss these during one of our plenary sessions or continue here, let us know.

Comment by @cwilso Oct 5, 2020 (See Github)

Yeesh, I somehow completely missed the notification on this response, sorry.

I think your (C) and (D) are completely reasonable, and I share those goals. On (A), of course we want the reviews to be useful, which is part of what pushed them later in the steps (when the ideas are better baked, rather than just a collection of use cases - please note that there is a definite goal of getting the Chromium process to encourage early engagement with the community, rather than waiting to drop a fully-formed idea of an API on the public community's plate. (Another way to read this is "the old de facto process was frequently well into prototyping (mid-late step 2) before it got announced publicly.)

The motivation for TAG review request being at the start of step 3 is that the entry to step 2 ideally doesn't have a design yet - you would be requesting a TAG review on a design that doesn't accomplish the goals, or is missing significant pieces. I'd already planned for if the TAG wants to be engaged when the ideas are first coming up, we can easily hook that in - we already have notification (at least in most cases) via the incubation path (the latter part of step 1 drives a WICG proposal, which will send new-incubation-repo notifications to public-review-announce@w3.org, which we could ensure is more systematic). I think that's a different request than the "there is a fleshed-out design that should achieve the goals, could you take a look" review, and that's probably what having a discussion would be good for.

The reason TAG review is mentioned in step 6, not step 4, BTW, is because it's a completion - if you haven't gotten a TAG review and responded to the feedback by the time you send an intent-to-ship, the API owners are going to be looking very askance. Perhaps it would be a good idea for me to add an explicit "you should really be wrapping up your TAG review!" to step 4?

If you have some time to discuss during your next plenary session, let me know (preferably not between 3 and 6 AM in Pacific time. :P )

The exit criteria for each step are captured in the Chromestatus tool, and

Discussed Oct 12, 2020 (See Github)

Peter: Chris did respond to our concerns - should we invite him to a meting...?

Rossen: I think we should discuss in a plenary session. Are there any changes on either side we can or should do?

Hadley: feels like there's a differene between when you would ideally do a TAG review vs when you would be in trouble if you haven't done one?

Rossen: also - chromium and w3c and TAG - how do the processes intersect?

Dan: So let's get him to a plenary session - maybe after TPAC week? Like 1st week of November?

Peter: maybe the 11th of November?

Peter: I will reach out to Chris about the 11th.

Comment by @plinss Oct 12, 2020 (See Github)

Hi Chris, thanks for the feedback. Can you join us for our plenary meeting on November 11th? The exact time of day may change as we adjust our scheduling for the DST change, but we're currently scheduled for noon Pacific (we'll keep it a PST friendly time if it moves).

Comment by @cwilso Oct 12, 2020 (See Github)

Sure! It would better for me if it were NOT between 12pm and 2pm on November 11th (I have a contractor doing work in my house at that time), but I could make it work either way.

Comment by @cwilso Oct 19, 2020 (See Github)

I left a sentence fragment above, completed my thought below:

The exit criteria for each step are captured in the Chromestatus tool, and the goal is to encourage earlier asking-for-review and notification steps, while not delaying progress (but explicitly asking for response to all review issues at appropriate times).

Discussed Nov 9, 2020 (See Github)

Guest was a no show.

Comment by @cwilso Nov 12, 2020 (See Github)

Gahh! I am so sorry, I didn't know when I was supposed to show up, and I think I missed it. My deep apologies; if we can get back on the calendar for this discussion, let me know.

Comment by @plinss Nov 12, 2020 (See Github)

Hi Chris, no problem, I also neglected to send out the meeting information in a timely manner.

Our plenary meetings alternate times to be more APAC friendly, our next meeting is at 10pm PST on Tuesday the 17th, otherwise I'd recommend 12pm PST on the 25th.

Comment by @cwilso Nov 20, 2020 (See Github)

12pm on the 25th?

Comment by @cwilso Nov 20, 2020 (See Github)

Also, I gave a (short!) talk on this at BlinkOn this week: https://youtu.be/hgEyQsy1D7w.

Discussed Nov 23, 2020 (See Github)

Peter: welcome, chris...

Chris: issue in design reviews... some good back-and-forth... first message is: the blink shipping process is continuing to be refined.

... we want to have TAG reviews be useful. not rubber stanps, but not delaying product schedules...

... we have early notices when incubation starts... we then send an email to public-review-announce@w3.org - PING uses this to pick up notification of new incubations... The developer trial - built enough code to be useful but under a flag - not in an origin trial. That's going to become a notification point to the dev community.

Dan: [lays out some of the questions]

Chris: not everyone who goes through this process is a long term standards participant -they might not understand what value TAG review gives. We don't want to give an easy out to skip TAG review.... A lot of the immersive features for example - get incunated in a community group - one of the vendors wants to implement and then goes for TAG review. WebRTC feaures - "do we need to get a TAG review on this" - maybe, because there haven't been a review in that area recently.

Dan: design principles, s&p questionnaire, explainer... how do we make sure these are being worked on early...

Chris: S&P questionnaire a good way to get useful guidance to people... people need to understand right points to ask for TAG review - and how to structure that ask as well. Early notiication of "we're gonna look at this problem space" and notification of dev trial - ... makes it more clear when to ask for a review...

Rossen: couple of points - some of the additional process being added - how do you guide engineers who have less experience in standards... Nobody reads the design guidelines... making engineers go

Dan: how do we make them more palatable? Seires of tik toks?

Rossen: yes people don't read those - they add value - people don't have the time to dig into them - creating summaries can help a lot. entertaining the idea of a check-list that is random. Gamify the system.... The reason why the security questionnaire works well is that it's short and it prompts people to think. Effort to create a questionnaire for the design principles would go a long way.

... is there an opportunity for early binding so the reviews don't turn into "looking backwards" and fixing lack of knowledge of the design doc, etc...

Chris: we have beefed up the standards mentor - any feature that you create you should get someone as a mentor to help you. Iterating on the chrome status tool ... walking people through that. One challenge is : varried amount of standards experience in the team working on chromium...

... what guidance can we give people about when to ask to TAG review and what expectation should you have about response?

... how to structure the guidance....

Rossen: my take is- the earlier the engagement the better. I've seen a lot of reviews marked as "early review" from the PoV of the enginner but these are not designs that are early. Sometimes the idea might not be a good fit. Ideation level of engagement - could be super-low-binding - not a design review but an ideation review...

Alice: explainers that are like a paragraph... might be good in some cases. moving the initial contact with TAG to step 1 ... there's a tension between "nothing to review" .. before they've done all the work... and writing a 2000 line ... Could make people defensive.

Rossen: also allows people to make connections early on...

Chris: dovetails well with revamp of the [blink] process... initial announcement is sometimes at the end of the feature design phase...

[discussion of development of toast]

"don't fall in love with your first good idea" <- attributed to dimitry glazkov

[raising issue of incubation... ideation level engagement...]

Chris: if we had a signal of incubation - TAG could pay attention to that...

... maybe we need a 2-stage review. New API design idea... the one-paragraph explainer.

... example of web bluetooth ...

Alice: chrome status is organized around features... we want to hear from folks who don't have a feature. in AOM we started with one idea, we refactored after feedback, now we have 5 separate things... Your first instinct would be "what API do I want to interact with" vs thinking the underlying need.

Alice: not asking the "what is th need" question is not good

Dan: we could hook into whatever incubation process when we are putting our own agendas together...

Peter: i would like to see something that triggers us to look at things... It should have a sniff test beforehand. Also we should not be blocking people. One of the ideas of the FYI review would be to look at them at a higher priority on the triage pass. Sometimes we want to give a signal : go ahead, but don't take it as "passes tag review".

Rossen: is the time that WICG repos get opened - those were not early ideation...

Chris: i'd prefer to get a little earlier - but they need to be structured enough so people think it's an interesting problem to solve...

... filing issues in a proposals repo to get new repos started...

Rossen: we need to make sure we're not going to be flooded with a firehose... Incubation/ideation; features that are new but not revolutionary; catch-up features. Catching up on features already shipped by others... the last one should be a no-op from a TAG PoV.

Dan: sometimes things get stalled... sometimes there is an impasse... how do we deal with that...

Alice: could there be an intergation with chrome status to the TAG review... to highlight that there is a stalled review?

Chris: we're revamping chrome status... not as obvious as it should be...

Rossen: how are "closed - not satisfied" reviews perceived? Does it change anything?

Chris: yeah - TAG review and it's oucome is one of the strong signals that API owners look at before deciding whether to ship a feature. Review from the TAG; partner feedback; otheer browser vendors; security & privacy review... a negative review from the TAG should result in it not shipping.

Alice: you can hook into our labels ... labels should give more insight.

Peter: wrap-up - consensus on 2 new types of reviews - fyi review for something small. Ideation review where we get in early... may be triggered on WICG repo being created or email to incubation mailing list...

Chris: we should have the early ideation engagement not only in TAG review...

Peter: we can use labels.... and pick it up with a bot.

Dan: we also took an action to work on a simplified / questionnaire version of the design principles... make it more accessible.

Comment by @plinss Nov 23, 2020 (See Github)

12pm on the 25th works. Join us at https://meet.jit.si/w3ctag

Discussed Feb 22, 2021 (See Github)

Dan: can we close this? Do we want continuous issues? We have a point of contact for Blink shipping process and have a process in place.

Peter: we had some actions to make additional issue templates for different phases of their shipping process, did we do that?

Rossen: that plus incorporate with early WICG stage proposal to surface early review. Other than that I agree with Dan, happy to close.

Dan: those are issues for process repo and work on it next week

Tess: there's a mailing list for whenever a repo gets created at WICG

Dan: creates issue to track this.

Discussed Sep 1, 2021 (See Github)

Dan: I think we need to close this one.. we have multiple ways of communicating with Blink.

Comment by @torgo Sep 16, 2021 (See Github)

Thank you @atanassov for opening this. We're going to close this as we have multiple ways in which we are communicating with Blink - e.g. liaison calls, a new label, etc... If we feel we are getting out of sync we can re-open and have further discussions.