Are we doing this right? Slow insert performance on indexeddb.

[michael-kitchin]

We're having difficulty achieving the performance we're looking for when bulk-inserting data into a local-only, indexeddb-backed PouchDB (v8.0.1), and at this point we're trying to determine whether we should expect what we're seeing or we're doing something wrong.

Our current observed performance inserting ~550k records is 1-2k insertions per second using bulkDocs() into an empty database (i.e., completely removed, via dev tools). This nets out to ~5min, when all is said and done.

Other specifics:

1. We're working on a ReactJS app written in typescript that we run either in the browser or Electron. Observed performance is roughly the same on both platforms.
2. Our dev boxes are all reasonably powerful (x86 running recent Ubuntu or Windows, recent-generation i7 or i9, 64GB+ of RAM, good SSDs). Our target systems are smaller, but we're already having problems on these dev systems.
3. We're inserting records that look like the following, where all keys and values are <15 characters: {"key1": ["val1"], "key2": ["val2"], "key3": {"key3.1": "val3.1", "key3.2": "val3.2"}}.
4. _id fields are prefixed UUIDs of the form 01234567890-f357fd5d-00ed-4b70-99fb-c512d9a75e35.
5. We've tried batch sizes from 100 to 10k with auto-compaction on/off and revs unlimited and limited to 1, and haven't noticed much difference (smaller batch sizes are a little worse).
6. Our specific batching scheme uses lodash _chunk() and promises wrapped up using Promise.all(). The "1-2k insertions per second" is based on batch timing, and therefore excludes database creation.

So what should we be expecting, do you think?

[alxndrsn]

@michael-kitchin is this using the idb adapter or the indexeddb adapter? Both use IndexedDB under the hood.

[jbeckton]

@michael-kitchin It would be interesting to see if your performance was any better using the in-memory adapter. 550K records is a lot of data for a browser application.