#72: Task Scheduling

Visit on Github.

Opened Sep 14, 2015

The CSS Houdini effort is looking to specify a pipeline model for execution of script in layout/raster: https://docs.google.com/document/d/1Mw6qNw8UAEfW96CXaXRVYPPZjqQS3YdK7v57wFttAhs/edit?pli=1#

This (re)raises the question about ordering for delivery of other tasks in the platform (Mutation Observer change records, Object.observe records, setTimeout callbacks, setIdleCallback callbacks, requestAnimationFrame callback delivery). This has been discussed at some length, e.g., with @wycats at TC39.

The open question here is if the TAG should try to untangle this and generate a recommendation for scheduling in the non-layout/raster bits of frame generation.

/cc @annevk

Discussions

Comment by @annevk Jan 13, 2016 (See Github)

As @domenic said, CSS should probably refactor https://html.spec.whatwg.org/#processing-model-8 to make things work for them. Review would be good, but I suspect for quite a bit of it we're bound by compatibility.

Comment by @travisleithead Mar 30, 2016 (See Github)

Probably need to enumerate all the timings in the system, then figure out what a "tick" might be, and describe what feature areas understand which ticks, which APIs slot into which ticks. Whether there might be paired responses (dependency control). Task re-ordering? Lots of things to consider.

Should also look at various frameworks (like fastdom) and understand what scheduling policies are in use.

Comment by @torgo Jul 30, 2016 (See Github)

Taken up at Stockholm f2f.

Comment by @dbaron Jul 30, 2016 (See Github)

What we (mostly Travis) wrote on the whiteboard during the meeting: img_20160730_114906

Comment by @torgo Apr 28, 2017 (See Github)

Scheduled for 5/16 to work through it. We agreed to work on it in the London F2F but that we would need to do some prep work ahead of time. /cc @travisleithead

Comment by @torgo Jul 26, 2017 (See Github)

Discussed at London F2F - possible we might create a note that "explains how this all works together."

Comment by @travisleithead Jul 26, 2017 (See Github)

Some thoughts during discussion in the breakout session:

Potential problem: no opportunity to process input during the rendering loop (which some consider higher-priority than rendering)? • Some implementations (i.e., Chrome) were considering processing input at the same rate as the Render-loop, but coalescing multiple input events of the same type (e.g., mouse/touch moves) into a single event (either dropping the others in the queue, or providing the additional data in the single event).

Invariants? What are they?

Background processing: (for things in background tabs, (or potentially in non-visible iframes, etc.) slowing the render-loop frequency (or enabling it to be time-sliced to process foreground work, can lead to increased queue sizes in the background tab, which can lead to 'jank' in the foreground tab when the background tab finally drains all the queues.

Where should certain processing be done relative to where layout is computed? Should be based on how the API might use layout info (e.g., wse layout data vs. set style data): • RequestAnimationFrame - Seems like purpose is to write styles (vs. reading layout), thus should ideally be done before computed layout (or you would run layout twice if layout was read in the callback) • IntersectionObserver - Purpose is to read layout, thus should be in the pipeline after layout has been done, if styles are written (then layout would be done twice). • ResizeObserver - Hmm. Writes styles and reads layout--where to put it?

Comment by @travisleithead Jul 26, 2017 (See Github)

We would want some guideance in our design-principles doc, about where a callback should slot into the Rendering Loop, or whether it should be a new task. See https://github.com/w3ctag/design-principles/issues/38

Comment by @travisleithead Jul 26, 2017 (See Github)

Pictorial view of HTML's render loop, with annotations for Edge noted in grey: image

Comment by @dbaron Jul 26, 2017 (See Github)

So a few further thoughts here:

  • anything that has callouts to script from within the rendering pipeline leads to the risk that we might have to repeat some of the core steps (e.g., style, layout) of the rendering pipeline within a single vsync / refresh tick
  • we tend to want to put the things whose purpose is primarily to write (e.g., requestAnimationFrame) early in the process, and things whose purpose is primarily to read (e.g., IntersectionObserver) late in the pipeline. When they're used the other way around, they can of course end up triggering repetition of steps.
  • When we end up doing things that require repeating some of the steps, do we run these things more than once? In most cases, probably not; we don't run the things that call out to script when there are layout or style flushes. However, in cases where the later steps whose primary purpose was expected to be reading end up instead making writes, we've now run other read steps with data that's now bad. Do we run them again? What if they keep looping around? (And how do they interact if there's more than one type of callout to script that we run again -- e.g., media query listeners on an iframe that poke style in their parent document and change the iframe's size, and say, an IntersectionObserver that does something similar?) I wouldn't be surprised if there's some pretty poor interop here.

Another way of thinking about the input processing issue is this: we want the rendering loop itself to be as small as possible so that we can get back to the event loop to do other things (e.g., processing input). It's also nice to tie some of the input processing to the refresh loop so that we can batch it and avoid unnecessary processing (e.g., mouse movement or touch coalescing). These two things seem to conflict a little bit, though maybe it's not a big deal. (Is it well-defined when microtasks and nanotasks run in response to any event dispatch that's tied to the rendering loop, or to any other callouts to user script?)

But, really, this issue is waiting for somebody to do the work of understanding and testing this.

(Sorry, this comment might not be completely intelligible, but we need to move on to a scheduled discussion in the meeting now...)

Comment by @domenic Jul 26, 2017 (See Github)

I just want to reemphasize that while it is good people are working to understand this section of the spec and implementations, I do not believe it's something that should end up in this design principles document. There are no design principles here to be had. Perhaps a blog post explaining things, or a pull request to the spec adding more non-normative explanatory text, or more tests to be written. But those are different.

Comment by @dbaron Jul 26, 2017 (See Github)

This issue isn't in the design-principles repo. It's the design-reviews (formerly spec-reviews) repo.

Comment by @domenic Jul 26, 2017 (See Github)

I guess I was over-extrapolating from

We would want some guideance in our design-principles doc, about where a callback should slot into the Rendering Loop, or whether it should be a new task. See w3ctag/design-principles#38

Comment by @torgo Sep 26, 2017 (See Github)

Taken up at Nice f2f.

Comment by @cynthia Sep 26, 2017 (See Github)

Found this, which might be the origin of the document in the initial issue post. https://github.com/atotic/event-loop

Comment by @dbaron Sep 26, 2017 (See Github)

The Houdini meeting where this was originally presented was minuted here: https://lists.w3.org/Archives/Public/public-houdini/2015Oct/0017.html

Comment by @torgo Sep 26, 2017 (See Github)

Large piece of work that the TAG isn't able to accommodate. Requires someone to do some archeology in multiple browser projects. Someone needs to do that work. @travisleithead to do something. @cynthia to look at webkit.

Comment by @jakearchibald Sep 27, 2017 (See Github)

Some other materials:

Comment by @travisleithead Jan 31, 2018 (See Github)

@travisleithead didn't make progress on this, nor did @dbaron, but both plan to make time before the next f2f.

Comment by @travisleithead Oct 30, 2018 (See Github)

TAG Paris F2F: another F2F, another lack of any forward progress here. I guess there's a reason this is one of our oldest issues... As noted earlier, this is not a F2F activity.

Discussed Dec 19, 2018 (See Github)

Travis: timing ... intersection observer ...

[david recently suggested writing tests]

David: Intersection observer issue led me to that one.

Travis: happy to participate in some time to collaborate on writing tests... Will reach out on the 14th

Comment by @dbaron Dec 19, 2018 (See Github)

One task I'd like to do here that might help move things forward a bit is write some tests to test how well different engines match the ordering in Update the rendering and maybe nearby steps.

Discussed Jan 15, 2019 (See Github)

David: We put this on the agenda today as a reminder to have me look into it.

Travis: I probably won't get to this by the F2F. Anyone want to take and drive this forward?

David: I can try to make some progress by the next F2F.

Comment by @torgo Jan 15, 2019 (See Github)

Bumped to the f2f to try to make some significant progress in a break-out group.

Comment by @dbaron Feb 5, 2019 (See Github)

I will try to make some actual progress on this around Feb 21, and we'll cycle back on the 26th.

Comment by @dbaron Sep 22, 2020 (See Github)

This is related to #489... which I was actually briefly confusing with this issue.

Comment by @dbaron Sep 22, 2020 (See Github)

So @atanassov and I looked into this in a breakout at our virtual "Cork" face-to-face meeting.

There's clearly still stuff to improve about the platform here. But we've had this issue open for five years, and it hasn't caused a lot of progress to happen (I did write one test, at least). So we think it probably makes sense to close this not because it's fixed but because it's unlikely to lead to further progress. (That said; there are some other issues that may lead to some progress; see previous comment.) And it's still possible that I or someone else will find time to look into some of the remaining issues.