#902: Observable API

Visit on Github.

Opened Sep 21, 2023

こんにちは TAG-さん!

I'm requesting a TAG review of Observables.

This proposal adds an .on() method to EventTarget that becomes a better addEventListener(); specifically it returns a new Observable that adds a new event listener to the target when its subscribe() method is called. The Observable calls the subscriber's next() handler with each event.

Observables turn event handling, filtering, and termination, into an explicit, declarative flow that's easier to understand and compose than today's imperative version, which often requires nested calls to addEventListener() and hard-to-follow callback chains.

Further details:

We'd prefer the TAG provide feedback as:

💬 leave review feedback as a comment in this issue and @-notify @domfarolino and @benlesh

Discussions

Discussed Dec 1, 2023 (See Github)

Ready to be worked on.

Comment by @artalar Dec 23, 2023 (See Github)

Hi! One more opinion: https://dev.to/artalar/my-criticism-about-the-new-observables-api-37d5

People in my bubble have had a bad experience with observables (especially with merging a few streams) and are scared of the API and the complexity that it could bring to the platform.

Comment by @benlesh Dec 26, 2023 (See Github)
  • It would introduce another way to accomplish the same thing, which would confuse beginners and make the platform more complex. We already have callbacks, promises, async/await, events, streams, and generators.

Observables are another way to accomplish a lot more than what current methods allow. So "sort of". Callbacks exist in everything else you listed. promises and async await are firmly different concepts and ECMAScript features, not related the DDOM. Events aren't really a construct, but rather something that can be handled by the other things you listed. Generators are wildly different than Observables, they're pull-based where observables are push-based. Async Generators are pull-then-push, and slightly more complicated than observables. Especially when you start talking about coroutines.

  • There will always be a shortage of operators. Definitely check out rxjs.dev/api.

IMO, as the lead maintainer of the library, RxJS has too many operators, and we've been deprecating and removing them over the years.

  • We are uncertain about how it should work. The popular observables library RxJS is constantly improving and changing. Its internal logic and codebase are not simple. So, why do we assume that the current proposed API will be sufficient for us in the future?

I'm very sorry, this part is just plainly false. We haven't added any new operators in years. We've removed a few operators, and a lot of the work that is going into RxJS over the last few years is aimed at simplifying the codebase. In general, outside of "pipeable" operators, RxJS has barely changed in the 10+ years of its existence.

The only thing that's really been evolving over this time period is the growing user base of the library. Despite not being funded by any major tech company. Despite not having a dedicated staff, and only being maintained by volunteers. Despite not having an advertising budget, people are choosing RxJS to help them manage coordinating events. And it's not just RxJS. If you checkout the README for the proposal, there are several wildly used projects employing an observable type in their codebase.

Discussed Jan 1, 2024 (See Github)

reassigned to Lea & Tess

Discussed Feb 1, 2024 (See Github)

Lea: Seems to have been validated by author pain points extensively, there are a ton of libraries for it. Does improve a part of the platform that does not currently have the best dx events. It does to events what promises did to callbacks. One minor point is I'm missing a discussion of how this compares to the xisting listener.. does add a property to event objects but doe sit have compelte parity with addeventlistener. Do authors still need addeventlistener for some things? I wouldn't immediately know how to do the same thing as the once option on addeventlistener. As a library author I'm missing discussion on how to create classes that leverage this in the best way. Right now if you want to create a library or a class in the best way that authors can listen to you have to create an event target which is suboptimal - I was wondering if this fixes that. Can library authors leverage observable so they don't have to extend event target. By looking at the explaienr I'm not sure. I can ask. Leaning positive.

Matthew: looks like someone is replying to something... the last comment is answers to questions asked previously, except I don't see those questions in this thread. The issue of really adding something new and different, or could this be duplicating stuff has come up. The things you specifically just highlighted haven't come up so I'd say it's worth asking.

Tess: Meta comment... I get worried when people are proposing a new feature whcih is a variant of an existing feature. We're doubling API surface. Two APIs which accomplish similar things might not be great. Counter example - xhr vs fetch - we couldn't adjust xhr to get to the future we wanted. It's an instructive example. The other concern I have is we can't get rid of the old thing. We can add this new thing which is a variant of the old thing, but it's critical when we do that kind of thing that the old thing and the new thing are defined in terms of a common model so it's clear how they relate and how they interoperate and hook into the underlying platform. If you don't do that you end up in a world of pain. We have enough pain on the web platform. The example from the past is the portal element... a variant of iframe... Maybe they've done all of this work. This is my gut reaction. The benefit of the new thing needs to be worth the cost of the api surface duplication. Maybe the answer is that they've done it and it's great.

Peter: this feels more like a wrapper around events. They're not replacing events, they're polishing the api surface for dealing with them. Which is fine. How does it work with once - my guess would be it's not designed to, this is for things that are going to happen over and over.

Lea: the once option in addeventlistening is not about things that only happen once, only things you want to listen to once. Shorthand to having a listener that you then remove

Peter: that's what I meant. If you only care about it happening once, why would you build an observable? It's not replacing a one time event listening, but something you want to be reacting to over and over

Lea: the idea that it has filtering built in... it seems to have a lot of quality of life improvements.. you might want to only listen once but only have all the filtering of targets and all of that that it supports

Peter: raises the question - do youw ant to keep listening until you find something that passes your filter, or just ignore if it doesn't..

Lea: use cases for both. Another concern - it seems that this is meant to be standardised in a venue that is not tc39. For something like this... we don't want to repeat the abort signal and abort controller situation. This should live on tc39. Kind of a problem that WHATWG is adding primitives that should live on ECMAScript.

Peter: this went through tc39 in the pat and was rejected.. but not necessarily in this form

Lea: what were the reasons? They might apply. addeventlistener carries a lot of baggage. This api should definitely have primitives that work together with existin events, but it should be independant of the DOM. Right now the only solution we have is that we can extend event target to apply to objects unrelated to the DOM. .... No question that the user need is there, but question is this the best solution.

Tess: about the venue .. also relevant

Peter: ... or are they purely one off? Also promises made thing scomposible.. also meant async and await to hide complexity but give all that power in a simple way. Does this have a similar path?

Tess: really highlights the importance of the venue concern. If this is a stepping stone towards a js language feature it really needs to be tc39..

Lea: worth trying to get right and not just ship something to be done with it. An observable designed well is something that will be used for decades. Should be done by the right venue. They do have a section about standards venue.

Tess: lots of interesting thoughts.. let's post lots of interesting comments.

<blockquote> </blockquote>
Comment by @LeaVerou Feb 13, 2024 (See Github)

Hi everyone, apologies for the delay @domfarolino.

We discussed this in a breakout today, though we ran out of time before reaching a conclusion. The following are some personal thoughts and comments.

First, the author pain points are clear, so motivation is pretty obvious.

My main concern is the standardization venue. It seems very clear that ES will eventually need an Observable primitive and we would not want the web platform to have another conflicting primitive. Also, when it comes to pure JS language features, TC39 is the clear fit for this. The fact that AbortSignal/AbortController were not standardized in TC39 is unfortunate, and shows in their ergonomics — I think it's important to avoid this pattern. You mention that Observable does to Events what Promises did to callbacks, but one of the reasons Promises were so successful is exactly that they were specced as a language feature.

Looking at https://github.com/tc39/proposal-observable/issues/201#issuecomment-1021145957 it appears that the primary issue stems exactly from AbortSignal / AbortController. FWIW although we generally recommend against monkey-patching, I think it would be the lesser of two evils to have the Observable API specced by TC39, and only a signal parameter elsewhere, than to have the entire Observable API specced by WHATWG or another browser-centric venue.

Another concern is that it seems that this still depends on the EventTarget infrastructure, and basically provides a wrapper over it with better DX. But if we go through the effort of creating a completely new API, it seems like it should not come with EventTarget baggage, an API that was designed for the DOM and later retrofitted to other objects. E.g. as a library author, I find it unfortunate that for my objects to support events that my users can listen to I need to make them inherit from EventTarget (and since we don't have multiple inheritance, this precludes them inheriting from anywhere else).

Stepping even further back, we have way too many different pub/sub patterns on the Web platform and DX suffers as a result. To an author it is unclear why we have an Events system and e.g. MutationObserver, ResizeObserver, etc. I understand the reasons, no need to explain them to me, I’m just saying most authors don't and from a UI perspective, it's an implementation detail that has leaked out into the UI because the DOM Events infrastructure did not support this use case well. For a new pub/sub pattern to win them all, we really need something that would be able to support all of these cases, and be used as the underlying model for all of them (i.e. DOM Events can become a special case of Observable, same with MutationObservers etc).

Lastly, as a more minor point than the architectural concerns above: is there 1-1 parity with what addEventListener can do? E.g. by looking at the API I could not figure out how to do an once event listener.

Also props for a nice explainer, and especially for having code examples! I also loved that some code examples had a collapsed version with how this is done currently — wish all of them had this!

Discussed Jun 1, 2024 (See Github)

Lea: has anyone been in contact with TC39?

Yves: I don't see any particular signal...

Lea: I can ping some TC39 folks and ask...

Yves: would be good to know if there are outstanding issues from their sides...

Lea: no webkit standards position... at least 3 tc39 folks have "thumbs-uped" my comment...

Yves: the comment on the webkit standards position - discussion between Anne and Dom ... discussed the idea of bringing this to a TC39 group.. a discussion in progress... https://github.com/WebKit/standards-positions/issues/292#issuecomment-2177303724

Peter: a whole section in their explainer about standards venue...

<blockquote>

Hi @domfarolino,

We looked at this today during a breakout. We do have consensus on most of the concerns I expressed above, especially around standards venue.

Furthermore, we had some questions about the API shape that we couldn’t figure out from the explainer, and we were hoping you might be able to clarify.

It seems (though we are not 100% certain) that this is a separate primitive than EventTarget and simply integrates with it. If that is correct, the current explainer is a little unclear on what the boundaries of each are. Does it use the same Event objects? Does it register event listeners behind the scenes? How does bubbling work? Does it support bubbling to parent objects when not using the EventTarget integration?

  • All examples are about listening to predefined events on existing objects. How do authors create objects that can emit events? Today they need to extend EventTarget which is suboptimal. Does Observable enable a new pattern for this?
  • Do you folks have any plans to integrate with any of the other pub/sub mechanisms on the web platform or JS runtimes? E.g. all the *Observer objects, or Node’s Event emitter?
  • While the code examples are certainly succinct, we had trouble figuring out how many of them worked and extrapolate how to write code about use cases not listed in the explainer. We were especially unclear on the arguments of subscribe() (what does next do? How does it differ from callback?)

We think a reworking of the explainer would really help answer these types of questions. Some (non-exhaustive) suggestions:

  • Start from use cases and user needs.

  • Include some simple code examples before the more pragmatic ones, so the reader can understand how the primitives involved work

  • Include code showing how authors can use this to enable their own objects to emit events.

  • A clear separation of what is Observable and what is EventTarget.

@littledan @ljharb @ctcpip @robpalme We saw you folks upvoted my last comment. Has there been any recent discussion in TC39 around this? Any plans to discuss it in the upcoming Plenary?

</blockquote>
Comment by @LeaVerou Jun 24, 2024 (See Github)

Hi @domfarolino,

We looked at this today during a breakout. We do have consensus on most of the concerns I expressed above, especially around standards venue.

Furthermore, we had some questions about the API shape that we couldn’t figure out from the explainer, and we were hoping you might be able to clarify.

It seems (though we are not 100% certain) that this is a separate primitive than EventTarget and simply integrates with it. If that is correct, the current explainer is a little unclear on what the boundaries of each are. Does it use the same Event objects? Does it register event listeners behind the scenes? How does bubbling work? Does it support bubbling to parent objects when not using the EventTarget integration?

  • All examples are about listening to predefined events on existing objects. How do authors create objects that can emit events? Today they need to extend EventTarget which is suboptimal. Does Observable enable a new pattern for this?
  • Do you folks have any plans to integrate with any of the other pub/sub mechanisms on the web platform or JS runtimes? E.g. all the *Observer objects, or Node’s Event emitter?
  • While the code examples are certainly succinct, we had trouble figuring out how many of them worked and extrapolate how to write code about use cases not listed in the explainer. We were especially unclear on the arguments of subscribe() (what does next do? How does it differ from callback?)

We think a reworking of the explainer would really help answer these types of questions. Some (non-exhaustive) suggestions:

  • Start from use cases and user needs.
  • Include some simple code examples before the more pragmatic ones, so the reader can understand how the primitives involved work
  • Include code showing how authors can use this to enable their own objects to emit events.
  • A clear separation of what is Observable and what is EventTarget.

@littledan @ljharb @ctcpip @robpalme We saw you folks upvoted my last comment. Has there been any recent discussion in TC39 around this? Any plans to discuss it in the upcoming Plenary?

Comment by @ljharb Jun 24, 2024 (See Github)

@LeaVerou I've been suggesting to @domfarolino ever since I became aware of the browser effort that TC39 would be a better venue, but there's not been much discussion in plenary about it, since imo unless Chrome is willing to wait for the language to add Observables, our hands are kind of tied.

Comment by @robpalme Jun 24, 2024 (See Github)

TC39 is available as a venue with no special strings attached.

@domfarolino I believe you already belong to an Ecma member company (Google) so it will be trivial for your lead delegate (Shu) to onboard you as a TC39 delegate. Discussion can happen at any time in Matrix or in adhoc meetings organised on the Reflector. And you would be welcome to schedule the item for formal discussion in the next meeting which is July.

Comment by @littledan Jun 25, 2024 (See Github)

One key reason why it would be difficult to do Observables in TC39 is its use of AbortSignal as an unsubscription mechanism--which is a good API pattern in alignment with the rest of the web platform. There are other features under discussion in TC39 which may benefit from this notion of cancellation, e.g,. Signals. I see multiple ways we can work this problem out layering-wise, given some more explicit collaboration between TC39 and WHATWG. I have recently raised this AbortSignal layering topic to TC39, and have started chatting with some people within WHATWG about it; I hope to follow up in the coming months with a more clear proposal (but if someone wants to take a stab at this sooner, feel free).

I'd suggest bringing Observables to discussion in TC39 simply for a review of its current shape, focusing mostly on what the feature is for and how it serves developers, rather than focusing on its venue. In particular, I want to make sure that there's a strong, shared understanding of the relationship between Observables and Signals (they are complementary in their use cases, and we should have a well-understood, shared story around them, even if they're standardized in different venues).

Comment by @domfarolino Jul 10, 2024 (See Github)

Response to the early review comment

For a new pub/sub pattern to win them all, we really need something that would be able to support all of these cases, and be used as the underlying model for all of them (i.e. DOM Events can become a special case of Observable, same with MutationObservers etc).

I think this is what Observables attempts to do! It is true the initial integration point we're working on is EventTarget#when()1 that returns an Observable over event listeners. But Observables are not, as you suggest, "dependent" on EventTarget (in that they are "limited" by EventTarget's dispatching semantics). Observables are generic; they need not vend Event objects, they can vend any kind of value. Because of this, we've thought of, and been open to, future integration with other APIs on the platform of the pub-sub sort.

Please see https://github.com/WICG/observable/issues/72 which mentions MutationObserver, ResizeObserver, and more as APIs with potential Observable-vending capabilities. I don't believe our proposal in any way precludes these kinds of APIs from converging on Observables.

Lastly, as a more minor point than the architectural concerns above: is there 1-1 parity with what addEventListener can do? E.g. by looking at the API I could not figure out how to do an once event listener.

Thanks for asking, this should be adequately described in https://github.com/WICG/observable/issues/66 and https://github.com/WICG/observable/issues/65. Basically, the parity exists in the operators and abort mechanism.

Also props for a nice explainer, and especially for having code examples! I also loved that some code examples had a collapsed version with how this is done currently — wish all of them had this!

Thank you very much! We tried to construct it in a way to where the most visually-useful information with a high bang-for-your-buck came first.

Response to newer review comment

It seems (though we are not 100% certain) that this is a separate primitive than EventTarget and simply integrates with it. If that is correct, the current explainer is a little unclear on what the boundaries of each are. Does it use the same Event objects? Does it register event listeners behind the scenes?

It is most definitely is a separate primitive from EventTarget. Every question here should be answered in https://github.com/WICG/observable?tab=readme-ov-file#the-observable-api. It shows a custom-made Observable that has nothing to do with EventTarget or Event objects, and highlights the supported conversion from arbitrary iterables/promises/async iterables ➡️ Observables. https://github.com/WICG/observable#:~:text=While%20custom%20Observables,Observer%20handler%20functions specifically mentions how event listeners are registered behind the scenes.

How does bubbling work? Does it support bubbling to parent objects when not using the EventTarget integration?

When used with EventTargets, Observables are wrappers around event listeners. So, bubbling works the same as it would with them, since that's all controlled by DOM event dispatching infrastructure that our spec does not touch. Bubble/capture isn't a concept for Observables that are not vended by EventTargets — it's only a concept for the EventTarget method that produces Observables, to tell the underlying event infra that the event listener backing the Observable is interested in one or the other.

  • All examples are about listening to predefined events on existing objects. How do authors create objects that can emit events? Today they need to extend EventTarget which is suboptimal. Does Observable enable a new pattern for this?

Some of this is already covered in https://github.com/WICG/observable?tab=readme-ov-file#the-observable-api, but our proposal does not provide a new way to emit events from within an EventTarget. It enables a new way to handle those events, by giving EventTargets the ability to vend Observables representing event listeners. Changes to EventTarget construction are outside the scope of our proposal.

  • Do you folks have any plans to integrate with any of the other pub/sub mechanisms on the web platform or JS runtimes? E.g. all the *Observer objects, or Node’s Event emitter?

This was mentioned above, but yes just for completeness, see https://github.com/WICG/observable/issues/72. I think lots of APIs have a potential future vending Observable objects.

While the code examples are certainly succinct, we had trouble figuring out how many of them worked and extrapolate how to write code about use cases not listed in the explainer. We were especially unclear on the arguments of subscribe() (what does next do? How does it differ from callback?)

Hmm, I was hoping https://github.com/WICG/observable#:~:text=The%20creator%20of,data%20is%20finished covered this sufficiently. Are the API shape / semantics really that opaque though? If so I suppose we could explain each input in more detail, although I would hate to duplicate too much of the IDL in the spec.

Standards venue

I personally think this is the least consequential topic regarding our proposal, and its review, so I'd like to not spend much time on it. For one, nothing precludes us from moving things into, or using things from, TC39 in the future if both communities really see this as absolutely necessary, as @littledan mentioned. AbortSignal/AbortController were standardized in WHATWG DOM, and there is some discussion today about being able to use these cancelation primitives from within TC39 world. That should be a solvable, backwards-compatible problem.

I'm not saying we'd design Observables with that path in mind. I'm saying that Observables and cancelation primitives have not gotten good footing in TC39, so they're being pursued elsewhere, and simultaneously they're not precluded from a future in TC39 world forever (should that eventually become necessary).

@LeaVerou I've been suggesting to @domfarolino ever since I became aware of the browser effort that TC39 would be a better venue, but there's not been much discussion in plenary about it, since imo unless Chrome is willing to wait for the language to add Observables, our hands are kind of tied.

I certainly hope we wouldn't have to wait another 8 years!!

@domfarolino I believe you already belong to an Ecma member company (Google) so it will be trivial for your lead delegate (Shu) to onboard you as a TC39 delegate.

Thank you, I will say though, Shu is supportive of our current path so I don't think we really need to make any sudden moves here. We're planning on standardizing it through WHATWG DOM.

Footnotes

  1. At least that's what I suspect the current on() method will turn into.

Comment by @ljharb Jul 10, 2024 (See Github)

To clarify, Observables' friction in TC39 was lack of implementer interest, from browsers at the time too; since that's changed, I see no reason it would have any difficulty in TC39 anymore - all it needs is someone to bring it back, and a good faith understanding that it wouldn't be shipped on the web outside of the staging process.

Cancelation primitives was a bit more complex and seemed more due to individual delegate misunderstandings/miscommunications than issues with TC39 itself. Also, by AbortSignal etc being designed outside of TC39, it ended up precluding itself from easy incorporation into the language, and it would be really unfortunate to repeat that mistake with Observable.