Welcome back to Post #2 of this Weird Web series. For those who found their way here outside of my LinkedIn, the idea is to explore unconventional (and occasionally rebellious) ways to use the internet.
In this ‘episode’ we’re examining how most browsers provide (free) data storage and sync mechanisms – that you can exploit to take your data anywhere.
[TL;DR: Watch the two videos below β they cover the key points of this post.]
πΌοΈ The Scenario
But first, let’s paint a scenario where such a mechanism might be of use:
1/4. You build yourself a simple To-Do/Kanban app.
2/4. You give it a silly name, in my case https://Kanban.Dog, because the .Dog TLD was cheap.
3/4. Since all the data (To-Do items) are typed into the app by you, and only you need to see them, you forego a remote database for storage. Instead you let the browser remember the data locally via LocalStorage.
** [LocalStorage is a feature available in all modern browsers that provides a small amount of disk storage per website. That way, you can persist your To-Do items when you close and reopen the site.]
4/4. Weeks later, you’re out and about – and realise your phone doesn’t show your tasks when you visit the site – because it’s not the device you’ve used thus far to make those tasks on. Your tasks are saved on one device – not all of them. They’ll still be there when you get home, but thatβs not very helpful right now.
Could there be a way to make it so that your data could be securely replicated and synchronised across different devices and browser installs, making an up-to-date experience available on all of them without a server?
π‘What About P2P? π€
We might jump to the conclusion that a direct browser-to-browser/P2P protocol might be useful here to help share data between devices.
WebRTC is currently the only browser protocol that enables direct P2P data communication. However, it requires a server for the browsers to trade connection metadata and instantiate their connections to one another. Even if we could dodge that requirement, there’s a fatal flaw with the P2P approach anyway: All browsers would need to be online at the same time to stay in sync.
Thatβs just not practical.
So we need some form of remote storage β but we want it to be:
browser-native
zero-maintenance
secure
free
So the real question becomes: Why don’t browsers provide a remoteStorage API that works just like localStorage, but syncs across devices/installs?
Well it turns out that ‘remoteStorage‘ is already a real draft spec. The idea in this draft is that the browser can let users give applications access to a data store. This storage could be hosted via 3rd party or self-hosted in a private cloud. The rights to access/modify data in the storage will be scoped per-domain, and there’s an OAuth flow to authenticate to the storage to provide a site with permissions.
It’s an interesting spec that may eventually become huge if browser and storage vendors (eg. Dropbox) support the initiative and make it easy for users.
However, saying “wait a few years” would be a very unsatisfying conclusion to this post – and it isn’t quite what we’re after anyway.
Well, as alluded to in the beginning, most browsers already have various mechanisms to store bits of data in the cloud and sync across devices – typically at least for Autofill, Passwords, and Bookmarks.
In theory, we just need to trick the browser into storing some data in one of these mechanisms from our app,and have some method to retrieve it again later. Let’s see how this might work.
π€ Using Autofill
Autofill and the password manager work similarly and likely share logic under the hood of most browsers. The key difference is scope: Autofill stores generic info like names, addresses, and credit cards for reuse across the web, while the password manager ties saved credentials to specific domains for security. Since our app data should be origin-bound, Autofillβs too loose a fit – so let’s skip it and explore how to stash our data in the password manager instead.
π Using the Password Manager
In my To-do app, all data is saved as a JSON string.
Suppose we were to chop that string up into small chunks, and spread that data over several/many stored passwords.
At first glance, this sounds like it would require tedious manual entry in your browserβs password settings – however we can leverage much of the built-in password handling UI/UX to make this process somewhat smooth.
Here’s the plan for a PoC:
In JavaScript we’ll chunk our data into bits suitable for browser passwords (e.g. Edge can remember passwords up to 256 characters)
We’ll then automatically generate a login form for each chunk, and pre-populate the password field with all chunk values – then dynamically add them all to the page.
When we hit submit across all of these forms, they’ll submit to the same origin of the application but we’ll force them to open as new tabs. Since the app will return status ‘200 OK‘, the browser will be convinced each submit was a successful login – and offers to save the password (chunk).
We need a way to keep track of which chunk is which if we’re going to reconstruct them. For this we can simply use the username field of each form to store the chunk index. Now each credential is a chunk index (username) and the chunk itself (password).
Loading in data in the future, on any device is simple, we create a fake login form within our application, and the user starts by ‘logging in’ as chunk_1. We prevent the default ‘submit’ behaviour of the form, and instead we simply read the credentials from the form and store them. The user then uses the same form to try to login as chunk_2, chunk_3 etc… until all data is loaded.
While a fun PoC and a technically valid approach – this method is pretty cursed from a user-experience and browser-compatibility perspective.
But can we do better? π
π Using Bookmarks
If youβve made it this far – good news. This oneβs actually useful.
Here’s the plan for a PoC:
Exactly as with passwords, let’s chunk our data into small segments. But instead append each chunk to a URL, i.e. https://kanban.dog?data={a chunk goes here}
We then bookmark all those URLs and now the data is “stored” in the browser – and, assuming it’s configured to, also somewhere in the cloud.
When you open these bookmarks, some JavaScript on the page can extract the parameter’s value/chunk and, hey presto, you’ve “read” it back again.
We get the user (me) to open all of the bookmarks for an app in their browser in succession when they want to resume their session, and then reconstruct all of the data – loading it back into local storage, one chunk at a time. You could also number the parameters to make reconstruction easier, ie. kanban.dog?data2={the second chunk goes here} – allowing users to read chunks/bookmarks ‘out of sequence’.
Here’s the Demo:
This is way more effective than you might expect.
Data Saving Pro’s
π Browsers often let you save all open tabs to a bookmark folder. So we could either open all bookmarks first (mouse middle clicks) – and then save them. Or drag each straight to a folder.
π If you’re thinking this will be a lot of bookmarks – and just a bit tedious – browsers can store bookmarks way longer than you’d expect:
Browser
Maximum URL Length
Chrome / Brave
2,083 characters
Firefox
65,536 characters
Safari
80,000 characters
Internet Explorer
2,083 characters
Edge
2,083 characters
For the current state of my To-do list, this all fits in a single URL/bookmark in my browser of choice.
Data Loading Pro’s
π On the second device, Browsers often let you open all bookmarks in a folder with one click.
π If your browser open a bookmark folder as a group, or can ‘close all tabs to the right’, you can likely now close them all in two clicks too.
π And now that your data is loaded, you just delete the folder of bookmarks. Ready to start the process again.
You could absolutely have a fairly merry time using bookmarks to sync arbitrary app-data among browsers.
π§ Conclusion
Good luck taking these proof-of-concepts to the extreme or uncovering new ones to mess with. I will, at-least, be adding bookmark-sync to a few apps.
This was a fun one to finally check off an old To-Do (long before kanban.dog existed), and itβs got me digging back into that dusty corner of my brain where half-baked ideas live.
Leave a comment