Weird Web Series: Packing Data onto Your Browser

Welcome back to Post #2 of this Weird Web series. For those who found their way here outside of my LinkedIn, the idea is to explore unconventional (and occasionally rebellious) ways to use the internet.

In this ‘episode’ we’re examining how most browsers provide (free) data storage and sync mechanisms – that you can exploit to take your data anywhere.

[TL;DR: Watch the two videos below β€” they cover the key points of this post.]

πŸ–ΌοΈ The Scenario

But first, let’s paint a scenario where such a mechanism might be of use:

1/4. You build yourself a simple To-Do/Kanban app.

2/4. You give it a silly name, in my case https://Kanban.Dog, because the .Dog TLD was cheap.

3/4. Since all the data (To-Do items) are typed into the app by you, and only you need to see them, you forego a remote database for storage. Instead you let the browser remember the data locally via LocalStorage.

** [LocalStorage is a feature available in all modern browsers that provides a small amount of disk storage per website. That way, you can persist your To-Do items when you close and reopen the site.]

4/4. Weeks later, you’re out and about – and realise your phone doesn’t show your tasks when you visit the site – because it’s not the device you’ve used thus far to make those tasks on. Your tasks are saved on one device – not all of them. They’ll still be there when you get home, but that’s not very helpful right now.

Could there be a way to make it so that your data could be securely replicated and synchronised across different devices and browser installs, making an up-to-date experience available on all of them without a server?


πŸ’‘ What About P2P? πŸ€”

We might jump to the conclusion that a direct browser-to-browser/P2P protocol might be useful here to help share data between devices.

WebRTC is currently the only browser protocol that enables direct P2P data communication. However, it requires a server for the browsers to trade connection metadata and instantiate their connections to one another. Even if we could dodge that requirement, there’s a fatal flaw with the P2P approach anyway: All browsers would need to be online at the same time to stay in sync.

That’s just not practical.

So we need some form of remote storage β€” but we want it to be:

  • browser-native
  • zero-maintenance
  • secure
  • free

So the real question becomes: Why don’t browsers provide a remoteStorage API that works just like localStorage, but syncs across devices/installs?

🧩 Where’s the remoteStorage API? πŸ€·β€β™‚οΈ

Well it turns out that ‘remoteStorage‘ is already a real draft spec. The idea in this draft is that the browser can let users give applications access to a data store. This storage could be hosted via 3rd party or self-hosted in a private cloud. The rights to access/modify data in the storage will be scoped per-domain, and there’s an OAuth flow to authenticate to the storage to provide a site with permissions.

It’s an interesting spec that may eventually become huge if browser and storage vendors (eg. Dropbox) support the initiative and make it easy for users.

However, saying “wait a few years” would be a very unsatisfying conclusion to this post – and it isn’t quite what we’re after anyway.

Well, as alluded to in the beginning, most browsers already have various mechanisms to store bits of data in the cloud and sync across devices – typically at least for Autofill, Passwords, and Bookmarks.

In theory, we just need to trick the browser into storing some data in one of these mechanisms from our app, and have some method to retrieve it again later. Let’s see how this might work.

πŸ”€ Using Autofill

Autofill and the password manager work similarly and likely share logic under the hood of most browsers. The key difference is scope: Autofill stores generic info like names, addresses, and credit cards for reuse across the web, while the password manager ties saved credentials to specific domains for security. Since our app data should be origin-bound, Autofill’s too loose a fit – so let’s skip it and explore how to stash our data in the password manager instead.

πŸ” Using the Password Manager

In my To-do app, all data is saved as a JSON string.

Suppose we were to chop that string up into small chunks, and spread that data over several/many stored passwords.

At first glance, this sounds like it would require tedious manual entry in your browser’s password settings – however we can leverage much of the built-in password handling UI/UX to make this process somewhat smooth.

Here’s the plan for a PoC:

  1. In JavaScript we’ll chunk our data into bits suitable for browser passwords (e.g. Edge can remember passwords up to 256 characters)
  2. We’ll then automatically generate a login form for each chunk, and pre-populate the password field with all chunk values – then dynamically add them all to the page.
  3. When we hit submit across all of these forms, they’ll submit to the same origin of the application but we’ll force them to open as new tabs. Since the app will return status ‘200 OK‘, the browser will be convinced each submit was a successful login – and offers to save the password (chunk).
  4. We need a way to keep track of which chunk is which if we’re going to reconstruct them. For this we can simply use the username field of each form to store the chunk index. Now each credential is a chunk index (username) and the chunk itself (password).
  5. Loading in data in the future, on any device is simple, we create a fake login form within our application, and the user starts by ‘logging in’ as chunk_1. We prevent the default ‘submit’ behaviour of the form, and instead we simply read the credentials from the form and store them. The user then uses the same form to try to login as chunk_2, chunk_3 etc… until all data is loaded.

Let me show you the PoC in action:

There are several issues with this approach:

  • πŸ’© The username/password length limitations in most browsers restrict the usefulness of this to only small data.
  • πŸ’© There’s some nuance between browsers in how they decide to trigger the offer to save passwords. For example many modern browsers will only offer to save passwords if the user explicitly interacted with the input fields of the form that triggered the submit – though you can hold tab and whiz through them all.
  • πŸ’© If you’re working on a statically hosted app which blocks POST requests, you’ll need to submit your password forms via GET. And browsers typically don’t offer to save passwords from a GET-based form (though Edge does).
  • πŸ’© There’s typically no way to remove all existing passwords for an origin without getting into a browser’s settings and doing it one-by-one. They normally do allow us to save updated passwords though – so we can overwrite previous chunks. However trying to ensure you have the latest chunks across all credentials becomes challenging and I’m yet to find an elegant solution to this.

While a fun PoC and a technically valid approach – this method is pretty cursed from a user-experience and browser-compatibility perspective.

But can we do better? 😈

πŸ”– Using Bookmarks

If you’ve made it this far – good news. This one’s actually useful.

Here’s the plan for a PoC:

  1. Exactly as with passwords, let’s chunk our data into small segments. But instead append each chunk to a URL, i.e. https://kanban.dog?data={a chunk goes here}
  2. We then bookmark all those URLs and now the data is “stored” in the browser – and, assuming it’s configured to, also somewhere in the cloud.
  3. When you open these bookmarks, some JavaScript on the page can extract the parameter’s value/chunk and, hey presto, you’ve “read” it back again.
  4. We get the user (me) to open all of the bookmarks for an app in their browser in succession when they want to resume their session, and then reconstruct all of the data – loading it back into local storage, one chunk at a time. You could also number the parameters to make reconstruction easier, ie. kanban.dog?data2={the second chunk goes here} – allowing users to read chunks/bookmarks ‘out of sequence’.

Here’s the Demo:

This is way more effective than you might expect.

Data Saving Pro’s

  • πŸ… Browsers often let you save all open tabs to a bookmark folder. So we could either open all bookmarks first (mouse middle clicks) – and then save them. Or drag each straight to a folder.
  • πŸ… If you’re thinking this will be a lot of bookmarks – and just a bit tedious – browsers can store bookmarks way longer than you’d expect:
BrowserMaximum URL Length
Chrome / Brave2,083 characters
Firefox65,536 characters
Safari80,000 characters
Internet Explorer2,083 characters
Edge2,083 characters

For the current state of my To-do list, this all fits in a single URL/bookmark in my browser of choice.

Data Loading Pro’s

  • πŸ… On the second device, Browsers often let you open all bookmarks in a folder with one click.
  • πŸ… If your browser open a bookmark folder as a group, or can ‘close all tabs to the right’, you can likely now close them all in two clicks too.
  • πŸ… And now that your data is loaded, you just delete the folder of bookmarks. Ready to start the process again.

You could absolutely have a fairly merry time using bookmarks to sync arbitrary app-data among browsers.

🧠 Conclusion

Good luck taking these proof-of-concepts to the extreme or uncovering new ones to mess with. I will, at-least, be adding bookmark-sync to a few apps.

This was a fun one to finally check off an old To-Do (long before kanban.dog existed), and it’s got me digging back into that dusty corner of my brain where half-baked ideas live.

See you in the next post.

-hiburn8

Leave a comment