TomWhat we talk about when we talk about storage for web
Hi there, Tom here! As a web dev, you for sure at least once have saved some chunks of data to the disk in your web application or, even better, PWA. Contrary to native apps and their richer set of APIs for persisting data, the web as a platform for our apps has its limitations when it comes to storing data, particularly large amounts. Let’s take a look at those and see how browsers handle storage differently as well as where each solution’s strengths and weaknesses are.
Know your browser
Regarding storage, Safari/WebKit (as of writing) has a quota of 1 GB for your web app’s stuff. If the threshold is reached, the browser will prompt the user to increase the amount of disk space your app can access.
Firefox grants 2 GB per origin, which is nice but also not huge. And Chrome and other Chromium-based browsers? It works as follows: Chrome (as the whole browser) can take up to 80 % of all disk space available, as that’s the way the program ist built. Out of these 80 % of total disk space, each origin, e.g. your web app, can consume up to 75 %. Here’s an example: Say your device has a total of 20 GB available. Chrome can use 80 % of it, that is 16 GB. And now your app is allowed ¾ of that, which equals 12 GB. Using my bleeding-edge slide ruler of choice, that means that a web app in Chrome/Chromium can be hungry to as much as 60 % of the total disk space available. Generous!
Now that we’ve talked about the basics, let’s take a look at some solutions.
Localstorage
Localstorage is a built-in, dead-simple key-value storage, where each operation is synchronous and each value a string - meaning everything not a string has to be stringified/parsed. B/c every call is synchronous, using localstorage therefore leads to blocking in your app. Don’t use this API for large or even medium or even small (you get the idea) chunks of data but only really tiny values such as flags or IDs. Furthermore, localstorage is not available in Web Workers, which is a nice segue to the next solution.
IndexDB
IndexDB is a true powerhouse when it comes to storing your data that is available in all modern browsers. It is a modern and high-performance key-value-storage, but with completely asynchronous calls that can be used both in main, the Service Worker as well as Web Workers. On top of that, IndexDB persists even blobs for you, if you wish so.
When working with this API, you’ll soon notice that it’s a low-level API with a learning curve. Enter RxDB.
RxDB
If reactive is what you’re looking for, this is the DB you’ll love. RxDB enables you to subscribe to changes in the DB (such as documents or even queries), a feature localstorage and IndexDB don’t support. RxDB at its core is built around adapters, enabling you to learn the API once and reuse it in various applications such as PWAs or React Native apps. For the web, IndexDB is being used as the adapter. Beyond the standard features you’d expect from a DB, RxDB can also sync up with your CouchDB- or GraphQL-endpoint of choice. On top of that, encryption for documents and key compression for reduced disk usage are supported as well. To use RxDB, you'll have to npm install it first.
PouchDB
Similar to RxDB without the full reactive nature, PouchDB has to be mentioned here as well. It too can sync with a CouchDB-endpoint for replication and has a rich set of query options. Note though that compound queries on large DBs can take to vast loading times! Test your document-schema in a large dummy-DB before setting on this solution.
Localforge
As the name suggests, localforge has an equal API-interface to localstorage, yet the main difference is that this DB is asynchronous and takes care of parsing the objects you provide, which saves you a little amount of time.
LokiJS (just a mention)
Ok, that’s not really a persistent DB, not at all. LokiJS is an in-memory-DB which I just wanted to mention in case you haven’t heard of it yet. Loki keeps everything in RAM, therefore enabling you some blazing fast performances for reads, writes and complex queries. This kind of DB is mainly used in servers that are up 24/7.
Verdict / Best practices for your web app’s storage
Surprisingly, there are quite a few DB solutions for your web app to choose from, yet the frontend isn’t fully matured when it comes to serious storage requirements. Browsers limiting you’re total space available to 1 GB (again, as of writing, let’s hope this changes soon) simply don’t allow for implementations equal to native apps. Only Chrome/Chromium enables you to persist large amounts of data to disk.
When it comes to libraries, the built-in ones serve you in both cases: quick and simple á la localstorage or high performance with a steeper learning curve and availability in Web Workers via IndexDB. Other solutions such as RxDB enable you to write reactive code on top of IndexDB’s advantages, which gives web devs a powerful tool for storing data. All in all, there are enough tools to choose from for you to write great apps!
- Tom