Questions!

IDBObjectStore.getAll(), IDBObjectStore.getAllKeys(), IDBIndex.getAll(), and IDBIndex.getAllKeys() look great. Glad they're being added!

The fact that these methods return an Array is interesting from the perspective of ES6 Iterators, Generators, and the protocol they expose. It seems like cursors are, in some way, a userland version of these (in much the same way that IDBRequest is sort of a user-land Promise). Has any thought been given to some form of unification and/or methods that return ES6 iterators (perhaps instead of the current Cursor)?

IDBObjectStore.openKeyCursor() looks good.

IDBTransaction.objectStoreNames similarly seems useful.

Cursor iteration is async, so we're awaiting guidance on Observables or whatever the latest thinking in ES-land is there. Since that's unsettled it seems premature to tackle that now.

Likely we'll tackle Promisifying the existing API first - e.g. via what's proposed at https://github.com/inexorabletash/indexeddb-promises - then tackle async iteration when the path forward is clearer.

Hey @domenic and @littledan: what's the latest on async iteration and generators and what would you recommend for this case? Reifying all the results seems a bit obnoxious if we don't need to.

It advanced to stage 2 at TC39 yesterday. My take on the room is that although the spec is done, a number of people in the room (@dherman most notably) would like to see it bake for a while in Babel and other places before advancing it. I can't recall any specific criticisms of the API though, just a general feeling of unease from a few members. The editor, @zenparsing, also made the good point that since async functions are still at stage 3, it's better to wait for those to get some implementations before trying to push async iterables too fast.

My take is that async iterables seem to be moving along pretty smoothly, and we have a pretty natural protocol proposed for it. It's nice to have this use case in mind in evaluating the proposal, which you can read about here: https://github.com/tc39/proposal-async-iteration . Streams do a pretty good job of exposing something that's sorta like async iterables, but more ergonomic to use directly, which can go and upgrade to them later, but waiting also seems like a fine strategy. Observables are still in more flux, with a pretty radical possible idea for doing them in terms of Promises and callbacks on the table.

Taken up on TAG telecon 20-07-2016.

@inexorabletash see our minutes here: https://pad.w3ctag.org/p/20-07-2016-minutes.md - is there any more recent info or thinking you can share with us on this? we'd like to get feedback sent out...

The specific methods mentioned in the issue shipped in Firefox and Chrome several months ago now, as the simple "give me a bunch of results" was such a clear performance win.

I'm continuing to look at integrating IDB with Promises, and Observables make sense for cursor-type operations. I don't think there is anything actionable here for the TAG at this time.

On Jul 20, 2016, at 2:32 PM, Daniel Appelquist notifications@github.com wrote:

@inexorabletash see our minutes here: https://pad.w3ctag.org/p/20-07-2016-minutes.md - is there any more recent info or thinking you can share with us on this? we'd like to get feedback sent out...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Oh! https://developer.mozilla.org/en-US/docs/Web/API/IDBObjectStore/getAll https://developer.mozilla.org/en-US/docs/Web/API/IDBObjectStore/openKeyCursor

According to the articles, these return IDBRequest (iterator), rather than an Array type.

Everything in IDB returns an IDBRequest, which populates the result property when the request is completed. (This is what happens when you implement an async API before Promises are added to the DOM, and don't have a time machine handy.) getAll() was specified to yield (via request property) an Array of results.

I think the comments here and in the TAG minutes hint at a handful of inter-related questions.

• Q: Why do we need getAll() versus some iterable type? A: That's what IDBCursor is for, but the performance (as designed) is poor for many scenarios since script must explicitly pull the next result via continue() (vs. a push-type async iterator)
• Q: Should it be an Array or some other type? Frozen? A: shipped as just Array. Could probably be made FrozenArray. Today, the result on IDBRequest - which can be any cloneable data - is not frozen (etc)
• Q: Do we want to be extending the API like this, vs. exploring async iterators A: This was such a common request/clear performance win that it made sense. In parallel, exploring adapting IDB to work with Promises (see https://github.com/inexorabletash/indexeddb-promises) and pursuing Observables for push-type async iteration makes sense as those firm up. We do need pull-type async iteration to model the current behavior of cursors, but there may be intermediate steps.

Q: Why do we need getAll() versus some iterable type? A: That's what IDBCursor is for, but the performance (as designed) is poor for many scenarios since script must explicitly pull the next result via continue() (vs. a push-type async iterator)

In parallel, exploring adapting IDB to work with Promises (see https://github.com/inexorabletash/indexeddb-promises) and pursuing Observables for pull-type iteration (i.e. adapting cursors) makes sense as those firm up.

Hmm, this needs serious discussion. As @littledan said upthread, observables are very speculative right now and async iterators are probably a better bet. I also don't understand why you think observables make sense for this use case or are faster: async iterators are strictly more powerful, and if you pull them as fast as possible they should get you results just as fast as a push solution would---with the added benefit that you can control your memory usage and consumption speed, whereas observables are little better than arrays in that they push things to you without any control over the flow rate.

I had an offline sync up with @littledan and understand the async iterator proposal better (I simply hasn't spent any time with it yet). Agreed it's a better fit for cursors. I'll be exploring it soon.

On Jul 20, 2016, at 3:12 PM, Domenic Denicola notifications@github.com wrote:

Q: Why do we need getAll() versus some iterable type? A: That's what IDBCursor is for, but the performance (as designed) is poor for many scenarios since script must explicitly pull the next result via continue() (vs. a push-type async iterator)

In parallel, exploring adapting IDB to work with Promises (see https://github.com/inexorabletash/indexeddb-promises) and pursuing Observables for pull-type iteration (i.e. adapting cursors) makes sense as those firm up.

Hmm, this needs serious discussion. As @littledan said upthread, observables are very speculative right now and async iterators are probably a better bet. I also don't understand why you think observables make sense for this use case or are faster: async iterators are strictly more powerful, and if you pull them as fast as possible they should get you results just as fast as a push solution would---with the added benefit that you can control your memory usage and consumption speed, whereas observables are little better than arrays in that they push things to you without any control over the flow rate.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

To clarify, I don't want to have an observable vs async iterator debate (neither is at Stage 3 yet), but I think async iterators, for one, would be a good fit here. As a transitional possibility before async iterators are there, it would be possible to do something analogous to streams and give cursors a .next() method which returns a Promise of an IterationResult. In my opinion, this is fairly clean to use directly, and it has a clear upgrade path to async iterables. I'm looking forward to hearing more from @inexorabletash 's exploration.

And also to clarify, I'd updated my comment after @domenic started replying (hence, my text and his quote differ). Cursors need a pull-type model, which is indeed a better fit for async iterators.

The proposal at https://github.com/inexorabletash/indexeddb-promises shows how cursors+promises already get a non-terrible syntax with the rest of the proposal (search for "fetch all keys in a range using a cursor") but aligning that with async iterators makes sense. I've got @littledan nearby if I have questions. (And feedback on that repo is welcome)

...

So... is there anything actionable here or should I close out this issue?

Taken up at Stockholm f2f(https://github.com/w3ctag/meetings/blob/gh-pages/2016/07-stockholm/agenda.md) and we agreed to close.