#346: Pointerevent extension
Discussions
Comment by @slightlyoff Mar 14, 2019 (See Github)
Any updates?
Comment by @NavidZ Mar 14, 2019 (See Github)
@slightlyoff any updates on our side or the TAG folks? I assume we provided all the information required. But let us know if there is anything missing.
Comment by @EiraGe Mar 21, 2019 (See Github)
Hi @alice, can we get an expedited review on this? We are waiting for the review to ship the predicted events api in chrome.
Comment by @annevk Mar 22, 2019 (See Github)
How does this feature interact with shadow trees?
What is the exact modification of the event dispatch algorithm required and where is the monkey patch specified?
The description of the two methods lacks the detail to lead to interoperable implementations around these questions, as far as I can tell.
Comment by @NavidZ Mar 22, 2019 (See Github)
@annevk can you explain a little more. You mean how does the predicted points interact with the shadow trees? What does this have to do with that at all? GetPredictedPoints API doesn't change anything at all regarding the dispatch algorithm.
Comment by @annevk Mar 22, 2019 (See Github)
@NavidZ say the target node is inside a shadow tree. How is the target attribute of the events returned by these two new methods updated to ensure the shadow tree does not leak?
Comment by @NavidZ Mar 25, 2019 (See Github)
@annevk this spec does not specify when to set those targets. It just guarantees that they are the same as the event's target. It is up to UAs when to set them. For example in Chrome we have this lazy approach that we only set this when it is accessed. But regardless of that, the guarantee that the target of the events returned by these methods are the same as the target of the dispatched event should prevent that leak. WDYT?
Comment by @annevk Mar 26, 2019 (See Github)
You need to define the actual architecture that makes that work. E.g., define that these events hold a pointer to some parent and redefine the target attribute's getter so it can sometimes return the some parent's target instead. That will make it possible to verify that it's safe and will ensure that other extensions don't clash with it. The actual implementation can still be different from that, as long as it's black-box indistinguishable.
Comment by @NavidZ Mar 26, 2019 (See Github)
@annevk I haven't seen something like that around. Would that be enough to just explain this logic in plain text in this spec? if not do you happen to have a similar example to this somewhere else so I can see what they did and use the same format?
Comment by @annevk Mar 26, 2019 (See Github)
I'm not sure what you mean. I'd expect this to require changes to the DOM Standard as well (e.g., you're proposing changes to Event
's target
attribute's getter).
It also seems this would potentially affect composedPath()
in weird ways (or perhaps that's forwarded too?).
A similar thing is how Event
objects have a relatedTarget internal slot that gets adjusted in various ways and other specifications defining subclasses of Event
use to define some of their features.
Comment by @NavidZ Mar 26, 2019 (See Github)
FYI @smaug---- @patrickhlauke
The coalesced events or predicted events aren't really dispatched events. We wanted to avoid defining a new structure to avoid extra complexity. So we used the same event structure for simplicity and defined a bunch of attributes in this spec to be no-op (such as stopPropagation) or return default value.
But at the same time I don't want to just add all sort of hooks which makes the specification all more complex. Do you think changing the type of these events to some other newly defined type is easier then than just adding all sort of hooks in the DOM spec? Obviously the new type will not have target or path() or composedPath or anything. Again the goal is to not over complicate this and add hooks in other specs that are only used by this single spec.
Comment by @annevk Mar 26, 2019 (See Github)
If these are only meant to carry some data using a dictionary would be a lot better, and likely easier, yes.
Comment by @NavidZ Mar 26, 2019 (See Github)
Yeah. Everything is just a value except offsetX/Y attributes which are calculated with respect to the target. So technically when the target of the dispatched event changes (like in shadow DOM cases) the offset of these coalesced/predicted points also needs to be recalculated with respect to the new target.
Comment by @annevk Mar 26, 2019 (See Github)
In that case it sounds like you still need some kind of more complicated setup. (But not that complicated. Objects have a pointer to the event and some of their getters do some calculations based on the Event
's internal target slot.)
Comment by @smaug---- Mar 29, 2019 (See Github)
I thought the idea was that one could handle coalesced events just like normal events, if needed, even if prototype was modified or so. But then, yeah, path isn't useful there anyhow.
Discussed
May 1, 2019 (See Github)
Alice: I had a read through it. I read through the issue thread as well. Anne had some questions about interaction with shadow dom, and some questions about the implementability of the spec. Does anyone else have context on pointer events?
David: IIRC, multitouch was original motivation. Redesign mouse events with knowledge that multitouch exists.
Sangwhan: competing with touch events, but it has issues. So we have 3 different things to capture pointer input.
Dan: Many loud advocates for pointer events approach that want it implemented more widely.
Alice: So, this didn't have an explainer. Looked at the demo, didn't have explanation. Was a bit frustrating. It adds some APIs into the PointerEvent
interface: getCoalescedEvents
and getPredictedEvents
. I don't know what predicted events are...
Dan: Maybe we should just push back that there's no explainer; we should ask for that before we do a review.
Alice: Open questions: what are predicted events for? (second question missed) Plus Anne's comments about implementability of the spec.
Alice: OK, I can comment with that.
(digression about knowing who's commenting on the issues)
(Lukasz leaves)
Sangwhan: I can see if I can disambiguate this -- been a long time since I touched pointer events in Presto.
Dan: Co-owner of issue with Alice?
Sangwhan: I can.
Dan: Postpone to when?
Sangwhan: This was expedited... so next week if possible? If Alice has time I could set up a separate call this week or next to discuss this.
Alice: next would be good
Dan: So next week, but bump to later if you want to
Comment by @alice May 9, 2019 (See Github)
Apologies for taking so long to get to this! We are in the process of trying to refine our processes to get quicker at working through our backlog, plus I missed the request to expedite.
Unfortunately I found this somewhat difficult to review. An explainer would help greatly - the spec has only minimal explanation of the problem these new APIs are intending to solve, which means I'm having to somewhat guess at the context, which also makes the review more time consuming for me.
In particular, I couldn't figure out from the spec what the purpose of getPredictedEvents()
is; the introduction only mentions getCoalescedEvents()
. When and why would I call this method? Are the predicted events computed on demand or are they pre-computed before the pointer event is fired? If the latter, is it possible to opt out to avoid unnecessary work being done? If the former, why is the browser better able to perform this computation than a javascript library, which has access to the same information via getCoalescedEvents()
?
Comment by @NavidZ May 13, 2019 (See Github)
@alice I have updated the introduction to explain the later addition of predicted events and pointerrawupdate to the introduction as well. I also cleared up the getCoalescedEvents explanation as well. Sorry that the introduction was missed as that section is probably going to be removed/rewritten once merged to the pointerevents specification.
Comment by @slightlyoff May 16, 2019 (See Github)
the demo link is a 404; perhaps this is the right location?
Comment by @EiraGe May 17, 2019 (See Github)
Yeah, sorry that I update the demo link recently....
Discussed
Jun 12, 2019 (See Github)
Alice: my feedback was asking for an explainer. My opnion is that the API looks fine - it's adding this pointerrawupdate event which is a spammy event - finegrained events on that. spec has comments like "don't handle this unless you have fast code handling it" - the getpredictedevents API is copying some platform ... I asked a question about why the browser needs to do this. Sangwhan pointed out that if the browser is doing it it happens off the main thread... Anne also had points about shadow dom.
David: Anne was considered that the spec was not formal enough to show it does the right thing in terms of shadow dom and that it would need to get moved into the relevant spec - pointer events and mouse events.
Hadley: I am finding it hard to engage with finding that it jumps into "how it works" and there is no explainer.
[many ideas of what it could be used for]
David: i've seen discussion - some of it comes down to a browser diffeence - some browsers get the mouse events from the OS and fire them through to the web app. what chrome does is it fires a mouse move per vsync cycle. That's probably a better model - but if you are doing a drawing application you actually want the set of coalesced mouse positions from the whole movement. So if you care about the path, you can get them.
Ken: also for gesture recognition.
Alice: and if you're a game... also some of this info is in the intro - but the text is hard to engage with.
Hadley: this issue made me realize that our trmplate does say "explainer/requirements doc/example code" ...
David: we did change that to explainer (containing user needs and example code)
Alice: in terms of this issue - the spec quality issues may be out of our remit...
Tess: that's the case of a lot of things that come our way...
Alice: i'm not saying we don't do the review...
Hadley: i think editorial is out of scope but if it's difficult to review it should be in scope.
Alice: it took more work - my vote would be to say "i think the API looks fine" - would give feedback that the spec was hard to read and we asked for an explainer but you didn't provide. the spec could be made clearer and provide some example code. The API looks fine though.
Lukasz: the spec has no security & privacy considerations - I'm worried that this would be giving versatile info about the pointer movement - and this ability to track mouse movements without any prompts - there are a lot of research works indicating some possibilities of user tracking based on mouse movements - i'm wondering whether this provides something not provided by old-style API. Should we ask them to fill in the security & privacy questionnaire.
Ken: i think it's fine - if people start using this for tracking this would slow down the whole browser so it could be a performance issue. Maybe there should be a prompt.
[discussion on performance-related issues - slowing down the browser]
Ken: can you write some of these comemnts, lukasz?
Lukasz: how often would the event fire?
Alice: (as david said) it's only chrome thay actually does this coalescing...
David: firefox does not, i'm not sure about pointer events....
Lukasz: i still feel there is a problem with old-style mouse tracking...
Alice: I'm not sure what the privacy issues are ...
Ken: I think it makes sense to ask them to cosndier...
Alice: they did fill in the s&p review - all "no" -
Ken: we could ask the question thay lukasz gave...
Dan: I don't think a "no no no" answer to S&P is appropriate considering the fine-grained info theu are making available.
Hadley: If we are looking at expanding the fingerprinting surface area, we should ask them to consider it themselves, and to put it in the spec so that implementers can take mitigating actions if they want.
Alice: Lukasz, can you file an issue on their repo?
Lukasz: OK, I will.
Alice: i will write a comemnt on the review talking about the spec quality - needs to be clearer.
Peter: also anne's comments about how if the raw/coalesced events would interact with shadow dom.
[meta-discussion on secondary call times
Comment by @lknik Jun 12, 2019 (See Github)
Comment by @patrickhlauke Jun 12, 2019 (See Github)
Thank you @lknik - we're hoping to merge the extension into what will become PE3 shortly, and will make sure to consider the implications for privacy/security of new features.
Comment by @lknik Jun 14, 2019 (See Github)
Sure. Any information about the plan for PE3 would be welcome, too. I still need to look at it all. In the meantime, if this is meant to be a self-contained review, perhaps adding the consideretions in would be a good idea anyway?
Discussed
Jun 19, 2019 (See Github)
Sangwhan: This was one of the "why is TAG so slow" complaints.
Sangwhan: I need another week on this.
Dan: Will revisit on the 26th.
Lukasz: Asked about privacy/security considerations. Got reply they're waiting for PointerEvents 3 (not entirely sure they intend to tackle it in the e. Are we removing privacy label from the current issue?
Comment by @NavidZ Jun 24, 2019 (See Github)
We believe we addressed major concerns from TAG in the latest version. There is also a brief explainer for the features in the repository. Feel free to file more issues against the PointerEvent repo if you want clarification on more stuff.
Discussed
Jun 26, 2019 (See Github)
Alice: we discussed this - sangwhan and I - an explainer did come in and fleshes [things] out reasonably well. We thought we are done.
Sangwhan: i had issues with the naming.
Alice: that coalesce is a tricky name for non-english-proficient... Spelling bee word. But might not be a frequently used API... I can't think of any good alternative.
Dan: bundled? Doesn't feel like we should block on that though.
Alice: a LGTM with suggestions.
Sangwhan: yes - i'm ok with that but I'd like to be on record about the english-language issue raised. When reading somneone else's code this won't read nicely.
Alice: I will write the feedback.
[consensus to close
Comment by @cynthia Jun 26, 2019 (See Github)
Thanks a lot! The explainer does make the intent and constraints a lot more clear. We'll hopefully discuss this in today's call.
Comment by @alice Jun 26, 2019 (See Github)
We discussed this in our telecon today and we're happy with the use case and the broad design here.
Happy to close this, but just some parting thoughts:
-
We have some mild concern that the naming around
getCoalescedEvents
uses a word which is uncommon and difficult to spell, and is likely to be difficult for folks whose English vocabulary is not advanced for whatever reason. Something likegetBundledEvents
might be easier for more people to understand. -
The spec text seems a bit rough, with a lot of long, hard to read paragraphs and some ReSpec warnings, and the editors would presumably want to clean it up at some point. We mention this because it made the review more difficult, and presumably implementers would also find it difficult to digest.
Thank you for your patience through this review process, and for your hard work on this API.
Comment by @NavidZ Jun 26, 2019 (See Github)
Thanks @alice.
Regarding the ReSpec issues and everything editorial we are going to fix them all. As a matter of fact this extension document is going to go away and we are going to merge the content one at a time to the main spec and keep everything fixed with no warnings/errors.
Regarding the naming of it, just to give a little background Pointer Events working group actually had all different names including the one you suggested if I recall correctly. We intentionally said we need choose a hard word so people don't just randomly use it as it is a very niche API and hopefully that makes them go and look it up to see what it does and what "coalescing" means really. We knew most of the web apps should not care about or use these points for their use cases and using them would just hurt their performance and we were hoping this hard and more accurate word would serve that purpose as well.
OpenedFeb 19, 2019
こんにちはTAG!
I'm requesting a TAG review of:
Further details (optional):
You should also know that...
[please tell us anything you think is relevant to this review]
We'd prefer the TAG provide feedback as (please select one):