#662: Gamepad API input events
Discussions
Discussed
Aug 23, 2021 (See Github)
Ken: this makes sense. They're not going to remove polling so you can still poll if you want. Getting events seems reasobable. One issue - button down or up - people are moving to a world where it depends how much the button is down, adaptive triggers - depends on how much or force feedback. Should some thought be given on how to support this as well?
Dan: kind of out of scope for this propoosal...
Ken: some considerations... a whole new API - or a buttonPressed event, buttonReleased event?
Yves: wouldn't it be better ... poll at regular intervals?
Ken: that's what they have today - it means if you're not good at polling enough then you lose event data.
Yves: if it's an analog value every small change in the pressure will release lots of different events. How do you filter? what is the granularity?
Dan: coalescing strategy for event mitigation
Ken: get coalesced events that returns the events combined
Yves: as multiple events?
Ken: you get one event that is the coalesced event, but you can get the original events
Yves: you can imagine you want the median value of ten different events that happen roughly at the same time?
Ken: that's what they give you by default. Some kind of medium.. join the events somehow. But then you can get the original events by calling getCoalescedEvents
Dan: the proposal is by google people. Scant information on additional support for this (e.g. in the ChromeStatus)
Yves: do they have specific events for accelerometers?
Ken: I don't know.. pretty sure they did something for VR
Yves: pretty sure accelerometers should be read every time because you want to know exactly what's going on
Ken: you don't want to coalesce accelerometers... draft doesn't seem to have anything on accelerometers
Dan: writes comment
Ken: does the event have a timestamp? In the gamepad interface.. why does that have a timestamp? Not part of the... [leaves comments]
Comment by @torgo Aug 24, 2021 (See Github)
Hi - we're just having a look at this today and on first pass it looks good. 👍 One question: what is the current status of any additional implementer support? Right now the ChromeStatus page says "unknown" for Safari and Firefox.
Comment by @kenchris Aug 24, 2021 (See Github)
Do the coalesced events contain a timestamp? I think that would be useful.
Also many modern controllers and event gaming keyboards can check the amount a button is pressed, and even the PS5 has adaptive triggers. Have you considered that use-case and how would you augment this API to support that? Maybe in that case buttonrelease and buttonpress makes more sense as names
Comment by @nondebug Aug 25, 2021 (See Github)
@torgo Chrome Status should have "Positive" for Firefox, referencing Ted's comment on the public-webapps mailing list. That's pretty old so I've submitted a standards position request and will update the status once that's resolved.
I haven't heard from Safari yet.
@kenchris, please take a look at this doc for our updated thoughts on how pressure-sensitive buttons and axis changes should be handled. The proposal in the explainer (called Proposal 1 in the doc) is probably not the most useful for developers since it makes it hard to get an accurate picture of the current gamepad state without adding unnecessary latency.
A typical gamepad is implemented using the HID protocol (or something like it) where all button and axis inputs are delivered in a single packet. If we fire separate events for each button and axis then it makes it harder to understand which change events are from the same update. For example, if you're handling joystick axes then you always want to handle the X and Y updates from the same update at the same time. If you try to handle X and Y updates sequentially, you'll trace out a stairstep pattern which is a poor representation of the actual joystick movement. It's possible to work around this, but it means you have to delay handling some inputs until you're sure you've received all the relevant events.
Pointer events avoid this issue by delivering X and Y axis updates in the same pointermove event. To follow the same pattern, we should combine all axis changes into a single axesmove event. And since some buttons can behave like axes, we should include them as well. Proposal 2 combines everything into a single event (we called it "change") that's fired on every input frame if there was any change to any button or axis.
Currently I'm leaning toward Proposal 3 which keeps buttondown/buttonup events along with the change event. It's rare that you would ever want to subscribe to all of them since the change event would include all the same information included in buttondown/buttonup but I think it does the best job of satisfying everyone's use cases.
buttonrelease and buttonpress makes more sense as names
Those seem reasonable and I'm open to changing the names. I prefer buttondown/buttonup because it keeps the API more consistent with mousedown/mouseup and pointerdown/pointerup.
adaptive triggers
DualSense adaptive trigger haptics, Xbox Impulse Triggers, Switch HD Rumble and other advanced haptics aren't supported through the Gamepad API. There's a Microsoft Edge proposal for a Haptic Device API that we're hoping we can adopt in order to support more advanced haptics.
Discussed
Sep 1, 2021 (See Github)
Dan: Taking at look at the response to our last comment. Looks good to me.
Rossen: they are coalescing everything by design in a single packet...
Comment by @kenchris Sep 15, 2021 (See Github)
Adaptive triggers is more than just controlling haptic feedback such as vibration. It is also about having non-binary values for buttons. Like a machine gun (in a game) might start shooting faster and faster the longer down you press the button.
Adaptive triggers can also allow developers to configure the tension the user will experience as they press the button:
Comment by @kenchris Sep 15, 2021 (See Github)
Other comments:
ongamepadchange
sounds like the gamepad itself is being changed. I think"onchange"
or"oninputchange"
would be better names.- If you have
oninputchange
, I don't think we should addonbuttondown
andonbuttonup
- It is better to teach developers to actually just use the other event. I don't thinkonbutton*
adds any convenience - but they might add confusion
Comment by @kenchris Sep 15, 2021 (See Github)
rawgamepadchange
is dispatched if any button or axis state has changed. When multiple changes occur at the same time, one event is dispatched that encapsulates all the changes.
A fourth
gamepadchange
event is proposed that is created whenever arawgamepadchange
event would be dispatched and placed into an internal queue. The event may be dispatched immediately or delayed for performance reasons. Queued events must be dispatched before animation callbacks are run and before abuttondown
orbuttonup
event is dispatched for that gamepad.
That is very complex and I would never have guesses that was the behavior from the event names.
Also, events cannot have side-effects so you would have to have the implementation be prepared to dispatch both events at all times. Listeners to one event type cannot change how events are being dispatched internally.
Are there strong cases to be able to configure how events are being dispatched? If not, please just choose one. If you really need it, then couldn't this be configured with a method instead, with a sane default.
Comment by @nondebug Sep 17, 2021 (See Github)
Thanks for the comments, @kenchris
Are there strong cases to be able to configure how events are being dispatched? If not, please just choose one.
There's a strong case to dispatch input events immediately to support streaming game clients. For cloud gaming, inputs need to be uploaded with minimal latency. The rawgamepadchange
event is intended to satisfy that use case and any other case where low latency is needed. Currently, the only "good" way to do this is to poll getGamepads()
as quickly as possible, which still adds a small amount of latency. Events tied to the animation frame loop aren't useful for this use case because they add latency, on average half the interval between animation frames which will always be worse than rapid polling.
The case for having a coalesced gamepadchange
event is that there can potentially be a huge number of rawgamepadchange
events. The input rate is determined by the gamepad hardware, not the user agent. If script can't keep up with the events then they'll queue and add latency. By flushing all events any time one event is fired, script is always processing fresh input data but still has access to all the historical input data since the last event. Synchronizing on the animation frame callback is convenient because that's where most applications currently process gamepad inputs.
If you really need it, then couldn't this be configured with a method instead, with a sane default.
I think configuring the dispatch behavior with a method on the Gamepad object could work. Probably gamepadchange
's coalescing behavior should be default so the user agent can control the event rate if needed. Changing from a coalescing mode to a non-coalescing mode should flush pending events to ensure no inputs are lost.
Is there any precedent for configuring the event dispatch behavior after an event listener is registered?
Also, events cannot have side-effects so you would have to have the implementation be prepared to dispatch both events at all times. Listeners to one event type cannot change how events are being dispatched internally.
I may have worded this confusingly, I want to specify the same behavior as in Pointer Events where "The user agent MUST fire a pointer event named pointermove when a pointer changes button state." The intent is to flush the gamepadchange
queue whenever a button's pressed
attribute changes regardless of whether any event listeners are registered.
If you have oninputchange, I don't think we should add onbuttondown and onbuttonup - It is better to teach developers to actually just use the other event. I don't think onbutton* adds any convenience - but they might add confusion
I agree, if the gamepadchange
overhead is tolerable then buttondown
and buttonup
don't add much. I think it's reasonable to drop them.
Adaptive triggers is more than just controlling haptic feedback such as vibration. It is also about having non-binary values for buttons. Like a machine gun (in a game) might start shooting faster and faster the longer down you press the button.
I may have misunderstood your original suggestion. Non-binary button values have always been supported by Gamepad API and in this proposal we'd fire a change event whenever a trigger value changes. I've only heard the term "adaptive triggers" used in the context of DualSense's adjustable tension.
I think you're suggesting buttondown
and buttonup
could be generalized in a way that better supports non-binary buttons. Instead of firing a single buttondown
when the trigger value crosses the button press threshold, we would fire multiple buttonpress
events as the value increases.
If we're dropping buttondown
and buttonup
then this is moot.
Discussed
Sep 27, 2021 (See Github)
Dan: long comment back from requester
Ken: [reads] .. they're asking [something].. I'm going to ask dominic. Feedback seems positive.
Dan: there's a PR.. happening in webapps but the explainer is still in a google doc, I'll ask them to bring the explainer over.
Ken: will add Anne
Dan: if we get feedback by tomorrow we could close at plenary. Other reason to keep it open?
Ken: wait and see. Want to see if Anne and Dominic think their comments.. maybe easy to close, let's wait.
Comment by @kenchris Sep 28, 2021 (See Github)
Is there any precedent for configuring the event dispatch behavior after an event listener is registered?
@domenic @annevk do you know of such a case, or do you have any other comments regarding this?
Comment by @torgo Sep 28, 2021 (See Github)
@nondebug one process issue - can you please move the explainer over to w3 space (webapps wg repo) along with the spec?
Comment by @annevk Sep 28, 2021 (See Github)
@kenchris it's not entirely clear to me what is being asked, but event listener registration is supposed to be side effect free, apart from performance optimizations. See https://dom.spec.whatwg.org/#observing-event-listeners in particular:
In general, developers do not expect the presence of an event listener to be observable. The impact of an event listener is determined by its callback. That is, a developer adding a no-op event listener would not expect it to have any side effects.
Comment by @kenchris Sep 29, 2021 (See Github)
Maybe we could do something like addEventListener("gamepadchange", callback, { coalesce: false })
Event listeners already have options like once: true
Discussed
Dec 1, 2021 (See Github)
Dan: Can we close this off?
Ken: My concern is .. it's unclear .. two different events but can't use both at the same time... Two modes of getting the events... coelesced and non-coalesced. Grey area... Should there be a start function like we have with generic sensors?
Ken: writes comment regarding side effects...
Comment by @kenchris Dec 7, 2021 (See Github)
I am not a fan of having two different event handlers when you are only supposed to use one of them and choose. Also, as they deliver data differently you can argue that they are not side effect free (as they configure this change) and that is thus against the web design principles (as Anne pointed out).
The Generic Sensor API works around this by having start()
and stop()
methods and start()
can take arguments, so you could do something like start( { coalesce: false })
Comment by @torgo Jan 30, 2022 (See Github)
Hi @nondebug any feedback on the above?
Discussed
Jan 31, 2022 (See Github)
[assigned rossen - will look again at the plenary]
Discussed
Feb 14, 2022 (See Github)
Dan: no feedback from requester. I can raise with Chris H tomorrow.
Comment by @nondebug Feb 16, 2022 (See Github)
Maybe we could do something like addEventListener("gamepadchange", callback, { coalesce: false })
I like this.
The Generic Sensor API works around this by having start() and stop() methods
With Generic Sensor you can create a new sensor object for each consumer but for Gamepad all consumers always see the same Gamepad object. We'd need to add a Gamepad.getGamepadSensor() method to return something that can be unique for each consumer so we can call its start() and stop() without interfering with other consumers. This could work but it seems overly complex to me.
Discussed
Feb 21, 2022 (See Github)
Dan: Got a response, but can't progress without Rossen. Plenary.
Discussed
Feb 28, 2022 (See Github)
Dan: The issue that Ken raised has not been.. not clear whether it should be a blocker. I think this sounds fine.
Sangwhan: some flaws in original gamepad API - they're aware of this - this is trying to fix some of them... 2ndary interface.
Sangwhan: ... I think this is fine.
Dan: Ken is comparing it to generic sensor API?
Sangwhan: Ken's idea would make things complex - for each event listener you would have differnt ways of handling things -- complicate implementation. I can understand both sides of the argument...
Dan: if it adds complexity.. The user need is articulated well. People want to play games.
Sangwhan: writing games with current web tech is not good, you have to implement a poll inside RAF. This fixes some of that because it lets you develop games in a way that web apps are supposed to be made, so the api is less horrible. I feel like this is good.
Dan: agree
Sangwhan: I'll say if it's a tradeoff with complexity it's okay for now, might want to revisit in the future
Discussed
Feb 28, 2022 (See Github)
'[from a]'
Punted to C for Sangwhan
Discussed
Feb 28, 2022 (See Github)
Rossen: Close?
Dan: pending external feedback. Need to be happy with response. Need Sangwhan's input.
Comment by @cynthia Mar 1, 2022 (See Github)
We'd need to add a Gamepad.getGamepadSensor() method to return something that can be unique for each consumer so we can call its start() and stop() without interfering with other consumers. This could work but it seems overly complex to me.
Understood. Thanks for clarifying - I think this is something that we may want to revisit if there are user needs in the future. For now, based on our last breakout discussion we think it's fine to move forward with this proposal as is.
Thank you for bringing this to our attention!
OpenedAug 4, 2021
Ya ya yawm TAG!
I'm requesting a TAG review of Gamepad API input events.
The Gamepad API requires applications to poll to detect new inputs. Many applications follow the spec's recommendation to coordinate polling with
requestAnimationFrame
. However, polling in this manner is likely to lose inputs since gamepads typically update more quickly than the animation frame rate. Polling will also increase average input latency by half the polling interval. This proposal addresses both the missed inputs and latency by adding input events that fire when the gamepad state has changed.Further details:
You should also know that...
Gamepads can potentially produce a lot of input events and we are interested in feedback on how to mitigate the spamminess of these events. Consider a DualShock 4 that has two 2-axis thumbsticks and updates at 1000 Hz. When both thumbsticks are moving it will generate 4 axis change events per input frame, or 4000 events per second. If the application allows events to queue then it will be handling stale inputs, adding input latency.
This is mostly an issue for analog axis inputs but also affects pressure-sensitive buttons and analog triggers.
We have some ideas for mitigating this:
See here for more discussion on this proposal, the above mitigations, and comparison with the earlier Firefox implementation of these events.
We'd prefer the TAG provide feedback as:
💬 leave review feedback as a comment in this issue and @-notify @nondebug and @jameshollyergoogle