Peeking into "local first" web development

#web # front-end # lo-fi

For a small note-taking app that I'm building to learn Elixir and Phoenix, I realized that it shouldn't have to rely on an internet connection to work. I wouldn't even use the app myself, if it required a internet connection to take notes.

Then I fell down the rabbit hole of local-first web development.

What is "local first"?

"local first" is an approach to developing web applications in such a way, that they could be used offline and the client retains a copy of their data. That also effectively eliminates many loading spinners and latency issues. Kind of like optimistic updates, but on steroids with PWA, Wasm and a bunch of other new abbreviations and acronyms.

The premise is that it's much easier to consider this behavior as the default and then expand, rather than the other way around. Somewhat similar to the mindset and approach of "mobile first" when it became a thing we started doing. And similarly, it also requires some mental shifts.

Much to my surprise, it's not a rare and obscure paradigm; it is actually getting a lot of traction and already has big players in the game.

Figma, Superhuman, and Linear are good examples of pioneering startups in the local-first paradigm and all rely on local-first ideas and bespoke sync engines to support their speed and multiplayer UX. Kyle Matthews in Some notes local first web development

I spent the majority of yesterday researching existing solutions, their differences, and what a vanilla approach would entail. After reading a bit into the problems involved, I quickly realized that doing this vanilla would be way harder than I would want it to be, and ditched any efforts in that regard and focused on existing solutions, for now.

I'm still at the beginning, and I don't intend to write an in-depth comparison or overview of the topic, at this point - or maybe ever. But I still wanted to share my results so far.

Common ingredients

These play a key role and come up a lot.

CRDT: Conflict-Free Replicated Data Type. It simplifies the "how to merge changes from multiple sources" problem and is also used a lot when dealing with collaboration in non-local-first applications. There's a lot of very math-y and academic-y literature around these. I have read none of them, yet.

IndexedDB: a low-level browser API for larger amounts of data storage in the browser, compared to WebStorage (localStorage/sessionStorage).

Also: SQLite, which I think is already widely known and I've used it a bunch myself. However, in local-first it is usually used as a WebAssembly build to persist data in the browser using IndexedDB or the Origin Private File System. Like wa-sqlite, sqlite-wasm or compiled to JS with sql.js. Wild!

Quick First Impressions

I had set myself a limit of 3 solutions to look closer at, and tried to make sure they didn't feel super-similar at first glance to each other. Eventually I settled on these:

There are a lot of actively maintained libraries in this area, that have a lot of overlap but focus on different things. These I will keep an eye on in the future, but didn't make my cut or didn't have a self-hosted option.

I will build at least a small prototype with those 3 solutions. All of them have good documentation, reactive bindings, and use TypeScript and have an ORM-like API or query builder.

ElectricSQL

https://electric-sql.com/

TinyBase

https://tinybase.org/

  • Has "lodash of local-first" swiss-army-knife vibes
  • Plays well with ElectricSQL
  • Tiny: 5.0kb-9.4kb.
  • Offers pre-built react components.

Evolu

https://evolu.dev/

  • Heavy focus on privacy and data-ownership
  • End-to-End encryption by default
  • Has a method for "no signup", "no login"
  • Provides a server for sync and backup among evolu clients

Resources that were helpful

ElectricSQL conveniently provides a list of literature on the topic and alternatives.

And localfirstweb.dev provides plenty of links to reading and video material and people and projects to follow. It's an entry point that almost all the libraries refer to.

Conclusion

I remember similar attempts from something like a decade ago, but under the banner of "offline first", with the likes of hoodie at the forefront. Sadly, hoodie has been abandoned, and I don't recall quite why. Their domain has been taken over by RxDB for growth hacking. RxDB is also one of the "local first" options and looks great but has a pricing model, that didn't meet my criteria.

Despite those early attempts, all of this still feels like a pioneering stage that has been really kicked off by new browser features like Wasm, as I haven't heard much talk about these in the media I consume. But I might be wrong about that, since at the same time, a lot of the solutions out there seem very mature, have been around for a while, and have been adopted by a lot.

Please let me know if you've worked with any of the libraries or have looked into this topic as well! I'm super curious to hear anyone's first-hand experiences with these via E-Mail or Mastodon. I'll share mine, when I have some, too!