#682: State extension for JS Self-Profiling API.

Visit on Github.

Opened Oct 23, 2021

Braw mornin' TAG!

I'm requesting a TAG review of State extension for JS Self-Profiling API.

Non javascript execution is hard to identify in traces captured with JS Self-Profiling API: From a trace we cannot differentiate idle activity from top level UA activity, so for example code that triggers asynchronous rendering work cannot be measured properly.

Further details:

  • I have reviewed the TAG's Web Platform Design Principles
  • The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG
  • The group where standardization of this work is intended to be done ("unknown" if not known): unknown

You should also know that...

We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify @cnpsc @acomminos

Discussions

Discussed Dec 1, 2021 (See Github)

Dan: so one feedback is that they jump right into talking about JS profiling without talking about the developer or user need.... A sentence such as "this is in service of making web apps more reliable" etc.. might be helpful.

Yves: can it be used to detect on the mobile if you are on battery saver mode... If you're profiling and you realise youre running slower than usual...

Dan: Fingerprinting issue?

Lea: seems like a stretch

Ken: Agree

Dan: Worth asking the question - have they considered how this could be used by fingerprinters and mitigation strategy...?

Ken: this is the extension of the existing profile API using markers... adding a marker to what's going on... so more detailed information... The developer marks what they're interested in .. they get a timestamp ..

Ken: not just scripting... also css, etc... system might work well on your super fast machine .. but might be very expensive layout on older devices...

Yves: if it's just markers then...

Ken: The sspec is called js self profiling but profiling things that are not javascript...

Lea: given that these markers are predefined the scope seems limited. At first I thought it was general purpose but doesn't seem to be the case...

Ken: you give the profiling a set of markers...

Dan: any cross-origin info?

Ken: no it's only for that tab - only for that origin.

Dan: seems like they need to be explain the mitigation a but more... leaves comment.

Ken: fine with the API... No difference.

Dan: it does enlarge of the scope ....

Ken: Definitely. Maybe it would be nice if ... it's good gor UAs ...

Comment by @torgo Dec 7, 2021 (See Github)

Hi folks! Regarding the mitigation against cross-origin leakage that you've described - can you expand on this a little bit and be a bit more clear about what the mitigation strategy is? Are there ways that you could envision this API being used to intentionally and maliciously leak data across origin boundaries? It would be good if you could elaborate on the potential misuse (and mitigations).

Comment by @kenchris Dec 7, 2021 (See Github)

I see that you mention this is good for identifying different behavior across UAs, but given that this will mostly be available on the same engine for a long time (Chromium) do you expect developers to correlate this data with other data, e.g. specific OS's, devices etc, like identifying that layout is a big issue in say India as it takes very long on a Moto G5 phone etc?

Discussed Feb 14, 2022 (See Github)

Dan: ken left a comment... no response so far...

assigns Sangwhan

Dan: this is about JavaScript performance. Explainer does not define the user need ...

Sangwhan: provenance - it's from a non-browser org... so this makes sense. Presumably they want to capture traces & profiles at the end user level instead of having to capture with devtools?

Dan: leaves comment

Comment by @torgo Feb 15, 2022 (See Github)

Hi @cnpsc we're just coming back to this in our call today. Can you give us an update on progress? We'd like to understand: what is the multi-stakeholder situation (has this been discussed with anyone outside of the Chromium community?) The explainer also does not start from the user need as we've tried to emphasise in our explainer explainer. What user needs is this in service of? Also would this go to Web Performance working group after incubation in WICG? Have you had any discussions with that WG?

Comment by @cnpsc Feb 17, 2022 (See Github)

Hi @torgo ! We've introduced the feature to the Web Performance working to get initial feedback, we had 2 presentations to the group including one at TPAC 2021. ( https://github.com/WICG/js-self-profiling/blob/main/doc/state-extensions-slides.pdf https://github.com/WICG/js-self-profiling/blob/main/doc/tpac-2021-slides.pdf)

For the user's need I used the explainer's template and put it in the goals' section https://github.com/WICG/js-self-profiling/blob/main/markers.md#goals Is this description too low level ? I can add a higher level description of the user's need to that paragraph then. We want to provide web developers with better insights in the browser activity during a profiler trace. Right now they are limited to only the scripting activity we is very limiting.

Comment by @torgo Feb 17, 2022 (See Github)

Hi @cnpsc yes it's describing the problem from the developer perspective rather than from the end-user perspective. It may seem like make-work but we're really trying to emphasize (in the TAG review process) the idea of end-user-centricity - in other words, that people who are working on new APIs for the web are thinking about new functionality from the end-user perspective. Also remember the explainer is not just for TAG review. For example - what you write in the explainer could help ti inform eventual developer documentation (e.g. on MDN). So yes please if you could start with the (end) user need that would be much appreciated. And thanks for the pointer to the discussion with the Web Perf working group - this is exactly the kind of "trajectory" information we are looking for when we review something in WICG or another community group.

Discussed Feb 21, 2022 (See Github)

Max: We need to wait for a response from requester

Sangwhan: did we do any technical analysis?

Dan: my comments were all about process

Sangwhan: I'll look at it

Discussed Feb 28, 2022 (See Github)

Max: waiting for the authors to respond...

Comment by @torgo Mar 1, 2022 (See Github)

@cnpsc @acomminos any update?

Comment by @torgo Mar 24, 2022 (See Github)

Just taking another look at this at our face-to-face.

I'm still concerned about the multi-stakeholder issue - ChromeStatus shows no additional implementer interest.

From @atanassov : What is the story around user consent – because essentially you are using cycles on the user's device. Should it be gated behind a permission?

From @cynthia : what about main thread CPU starvation caused by a worker or an external process? Also ProfilerTrace and others should be annotated as serializable. Also why a document policy over a permission policy?

Comment by @cnpsc Mar 30, 2022 (See Github)

Thanks for the feedback, I tried to address the comments below !

What is the story around user consent – because essentially you are using cycles on the user's device. Should it be gated behind a permission?

For the user consent story, this has been explored in the original tag review for the feature, reasoning is that this is a source of bias in the performance profile and it limits ability to profile first load. See https://github.com/w3ctag/design-reviews/issues/366#issuecomment-496668592

what about main thread CPU starvation caused by a worker or an external process?

External processes are out of scope, the goal is to help developers understand how their own application behave in the wild. This include hotspots and unecessary code execution wasting the user's CPU cycles.

ProfilerTrace and others should be annotated as serializable.

Thanks adding the annotations !

Also why a document policy over a permission policy?

AFAIK at the time there was no 'disabled by default' support for Feature/Permission policy when this was added to the spec. Also Document Policy header serves as very early signal to UA's JS engine that the document has intent to profile and start storing the necessary metadata.

Discussed Apr 4, 2022 (See Github)

Dan: comments recently..

Sangwhan: will re-ask my question..

Dan: they didn't respond about multistakeholder

Comment by @cynthia Apr 5, 2022 (See Github)

Thank you for the response. Just to not lose context here - is there multiple stakeholder interest for this?

External processes are out of scope, the goal is to help developers understand how their own application behave in the wild. This include hotspots and unecessary code execution wasting the user's CPU cycles.

What about workers? The main concern was about the main thread profile data missing context about high compute pressure, caused either by a worker or an external process. It would be useful if there was a way to share some context when main thread is not getting enough CPU time, as the perf timings for those scenarios would be very different.

Discussed Apr 11, 2022 (See Github)

Dan: Sangwhan left a comment. Looks like not much update since then.

Max: We had a discussion, the authors didn't respond yet

Dan: Plenary? Wait to next week for Sangwhan

Discussed Apr 18, 2022 (See Github)

Amy: looks like waiting for response

Comment by @cnpsc May 2, 2022 (See Github)

Hi @cynthia, this development has been paused to gather more stakeholder feedback.

Thanks !

Discussed May 23, 2022 (See Github)

Sangwhan: I think this is pending because it's paused at the moment. We can probably close this. [will comment]

closed

Comment by @cynthia May 24, 2022 (See Github)

Thanks @cnpsc, feel free to request a re-open once you feel like you have reached a point where this warrants a review.