my blog https://overreacted.io

alt

Changed files
+32 -32
public
open-social
+32 -32
public/open-social/index.md
··· 11 11 12 12 I believe we are at a similar juncture with social apps as we have been with open source thirty five years ago. **There's a new movement on the block.** I like to call it *"open social"*. There are competing visions for what "open social" should be like. I think the [AT Protocol](https://atproto.com/) created by Bluesky is the most convincing take on it so far. It's not perfect, and it's a work in progress, but there's nothing I know quite like it. 13 13 14 - ![](./0-full.svg) 14 + ![A web of data: Alice's and Bob's records link to each other.](./0-full.svg) 15 15 16 16 In this post, I'll explain the ideas of the AT Protocol, lovingly called *atproto*, and how it changes the relationship between the user, the developer, and the product. 17 17 ··· 31 31 32 32 Or you type `https://bob.com` and you end up on Bob's website. 33 33 34 - ![](./1-full.svg) 34 + ![Alice and Bob have top-level domains that serve some HTML.](./1-full.svg) 35 35 36 36 In a sense, your browser is a portal to millions of different worlds, each with its own little jurisdiction. Only Alice decides what appears on Alice's website. Only Bob decides what appears on Bob's website. They meaningfully "own their data". 37 37 38 - ![](./2-full.svg) 38 + ![Alice and Bob meaningfully control those domains.](./2-full.svg) 39 39 40 40 This doesn't mean that they're isolated. On the contrary, Alice can embed Bob's picture with an `<img src>`, and Bob can link to Alice's page with `<a href>`: 41 41 42 - ![](./3-full.svg) 42 + ![HTML hosted by Alice and Bob can link to each other.](./3-full.svg) 43 43 44 44 Alice and Bob can link to each other, but they remain in charge of their sites. 45 45 46 46 What do I *mean* by saying Alice and Bob are in charge of their own sites? Even if they're not physically hosting their content on their own computers, they could always change hosting. For example, if Alice's hosting provider starts deleting her pages or injecting ads into them, Alice can take her content to another host, and point `https://alice.com` at another computer. *The visitors won't need to know.* 47 47 48 - ![](./4-full.svg) 48 + ![Alice can point her domain to a different hosting with her existing content.](./4-full.svg) 49 49 50 50 **This is important.** Hosting providers have no real leverage over Alice and Bob. If the hosting provider "turns evil" and starts messing with your site, you can just walk away and host it elsewhere (as long as you have a backup). You're not going to lose your traffic. All existing links will seamlessly resolve to the new destination. 51 51 52 52 If Alice changes her hosting, Bob won't need to update any links to Alice's website. Alice's site will keep working as if nothing had happened. At worst, a DNS change might make it inaccessible for a few hours, but then the web will be repaired: 53 53 54 - ![](./3-full.svg) 54 + ![Alice's and Bob's sites still link to each other, like nothing has changed.](./3-full.svg) 55 55 56 56 Imagine how different the incentives would be if links *were* tied to physical hosts! 57 57 ··· 71 71 72 72 These entities are usually stored in a database on the social company's servers. The most common way to visualize a database is as a sequence of rows, but you could also visualize it as a graph. This makes it look very similar to web itself: 73 73 74 - ![](./5-full.svg) 74 + ![Alice and Bob have pieces of data (a like, a profile, a follow) instead of pieces of HTML.](./5-full.svg) 75 75 76 76 What does this social graph enable that a web of personal sites doesn't? 77 77 ··· 83 83 84 84 Today, the most common way to implement these features is shaped like this: 85 85 86 - ![](./6-full.svg) 86 + ![Alice's and Bob's data are actually in a box, which represents a database. There are projections from that box to different application routes and screens.](./6-full.svg) 87 87 88 88 There still *exists* a web-like logical model of our data--our profiles, our posts, our follows, our likes, all the things that we've created--but it lives *within* some social app's database. What's exposed to the web are only *projections* of that model--the Home screen, the Notifications screen, the HTML pages for individual posts. 89 89 ··· 91 91 92 92 However, something got lost in the process. *The web we're actually creating*--our posts, our follows, our likes--is no longer meaningfully ours. Even though much of what we're creating is public, it is not a part of the open web. We can't change our "hosting provider" because we're now *one step removed* from how the internet works. We, and the web we create, have become rows in somebody else's database: 93 93 94 - ![](./8-full.svg) 94 + ![Nothing is meaningfully Alice's or Bob's anymore. They live within boxes controlled by companies like Facebook and Twitter.](./8-full.svg) 95 95 96 96 This creates an imbalance. 97 97 98 98 When Alice used to publish her stuff on `alice.com`, she was not tied to any particular hosting provider. If she were unhappy with a hosting provider, she knew that she could swap it out without losing any traffic or breaking any links: 99 99 100 - ![](./4-full.svg) 100 + ![Alice is moving content to another host.](./4-full.svg) 101 101 102 102 That kept the hosting providers in check. 103 103 104 104 But now that Alice publishes her stuff on a social media platform, she can no longer "walk away" without losing something. If she signs up to another social platform, she would be *forced* to start from scratch, even if she *wants* to retain her connections. There is no way for Alice to sever the relationship with a particular app without ripping herself, and anything she created there, out of its social graph: 105 105 106 - ![](./10-full.svg) 106 + ![Alice can't leave Facebook without erasing herself and her content out of existence.](./10-full.svg) 107 107 108 108 The web Alice created--who she follows, what she likes, what she has posted--is trapped in a box that's owned by somebody else. To leave it is to leave it *behind*. 109 109 ··· 115 115 116 116 Maybe the app gets squeezed by investors, and every third post is an ad. Maybe it gets bought by a congolomerate that wanted to get rid of competition, and is now on life support. Maybe it runs out of funding, and your content goes down in two days. Maybe the founders get acquihired--an exciting new chapter. Maybe the app was bought by some guy, and now you're slowly getting cooked by the algorithm. 117 117 118 - ![](./11-full.svg) 118 + ![What used to be Twitter is suddenly a different site.](./11-full.svg) 119 119 120 120 If your next platform doesn't respect you as a user, you might try to leave it, too. 121 121 ··· 135 135 136 136 Something has changed, though. (Can you spot it?) 137 137 138 - ![](./12-full.svg) 138 + ![Alice and Bob have pieces of data, like before. But Alice's username is `@alice.com` rather than `@alice`, and Bob's username is `@bob.com`.](./12-full.svg) 139 139 140 140 Notice that Alice's handle is now `@alice.com`. It is not allocated by a social media company. Rather, her handle is *the* universal "internet handle", i.e. a domain. Alice *owns* the `alice.com` domain, so she can *use it as a handle* on any open social app. (On most open social apps, she goes by `@alice.com`, but for others she wants a distinct disconnected identity, so she owns another handle she'd rather not share.) 141 141 ··· 145 145 146 146 When you previously saw the social graph above, it was trapped *inside* a social app's database. There was a box around that graph--it wasn't a part of the web. With open social, Alice's data--her posts, likes, follows, etc--*is* hosted on the web itself. Alongside her personal site, Alice now has a *personal repository* of her data: 147 147 148 - ![](./13-full.svg) 148 + ![Alice's website contains Alice's HTML. Alice's repository contains Alice's data. They exist side by side.](./13-full.svg) 149 149 150 150 This "repository" is a regular web server that implements the [AT Protocol](https://atproto.com/) spec. The only job of Alice's personal repository is to store and serve data created by Alice in the form of signed JSON. Alice is technical, so she likes to sometimes inspect her repo using open source tools like [pdsls](https://pdsls.dev/), [Taproot](https://atproto.at/), or [atproto-browser](https://atproto-browser.vercel.app/). 151 151 ··· 153 153 154 154 Have another look at this picture: 155 155 156 - ![](./12-full.svg) 156 + ![Alice and Bob have pieces of data.](./12-full.svg) 157 157 158 158 **These aren't rows in somebody's database. This is a web of hyperlinked JSON.** Just like every HTML page has an `https://` URI so other pages can link to it, every JSON record has an `at://` URI, so any other JSON record can link to it. (On this and other illustrations, `@alice.com` is a shorthand for `at://alice.com`.) The `at://` protocol is a bunch of conventions on top of DNS, HTTP, and JSON. 159 159 ··· 167 167 168 168 If Alice is unhappy with her hosting, she can pack up and leave: 169 169 170 - ![](./14-full.svg) 170 + ![Alice seamlessly moves her repository data to a different host.](./14-full.svg) 171 171 172 172 *(This requires a modicum of technical skill today but it's getting [more accessible](https://pdsmoover.com/info.html).)* 173 173 174 174 Just like with moving a personal site, changing where her repo is being served from doesn't require cooperation from the previous host. It also doesn't disrupt her ability to log into apps and doesn't break any links. The web repairs itself: 175 175 176 - ![](./12-full.svg) 176 + ![Alice's data is still Alice's. Bob's data is still Bob's. The links between them still work.](./12-full.svg) 177 177 178 178 It is worth pausing for a moment to appreciate what we have here. 179 179 ··· 187 187 188 188 When you make a post on [Bluesky](https://bsky.app/), Bluesky puts that post into *your* repo: 189 189 190 - ![](./15-full.svg) 190 + ![The Bluesky app calls createRecord in a repo. A post appears.](./15-full.svg) 191 191 192 192 When you star a project on [Tangled](https://tangled.org/), Tangled puts that star into *your* repo: 193 193 194 - ![](./16-full.svg) 194 + ![The Tangled app calls createRecord in a repo. A star appears.](./16-full.svg) 195 195 196 196 When you create a publication on [Leaflet](https://leaflet.pub), Leaflet puts it in *your* repo: 197 197 198 - ![](./17-full.svg) 198 + ![The Leaflet app calls createRecord in a repo. A publication appears.](./17-full.svg) 199 199 200 200 You get the idea. 201 201 ··· 203 203 204 204 To avoid naming collisions, the data in the repository is grouped by the format: 205 205 206 - ![](./18-full.svg) 206 + ![Alice's repo contents, separated by record type.](./18-full.svg) 207 207 208 208 In any user's repo, Bluesky posts go with other Bluesky posts, Leaflet publications go with Leaflet publications, Tangled stars go with Tangled stars, and so on. Each data format is controlled and evolved by developers of the relevant application. 209 209 ··· 215 215 216 216 That might remind you of [Gravatar](https://gravatar.com/), but it works for *every piece of data*. Every open social app can take advantage of data created by every other open social app: 217 217 218 - ![](./19-full.svg) 218 + ![Alice's repo content, this time without separation. Content created by each app flows into every app.](./19-full.svg) 219 219 220 220 There is no API to hit, no integrations to build, nothing to get locked out of. All the data is in the user's repository, so you can parse it (as typed JSON), and use it. 221 221 ··· 239 239 240 240 I've previously used a CMS as an analogy--for example, a blogging app could directly write posts to your repository and then read posts from it when someone visits your blog. This "singleplayer" use case would not require aggregation at all. 241 241 242 - ![](./20-full.svg) 242 + ![The Leaflet app requests data from the user repository.](./20-full.svg) 243 243 244 244 To avoid hitting the user's repository every time you want to display their blog post, you can connect to the user's repository by a websocket. Every time a record relevant to your app is created, updated, or deleted, you can update your database: 245 245 246 - ![](./21.svg) 246 + ![The Leaflet app subscribes to updates from the user repository by websocket.](./21.svg) 247 247 248 248 This database isn't the *source of truth* for user's data--it's more like an app-specific cache that lets you avoid going to the user repo whenever you need some data. 249 249 ··· 253 253 254 254 To avoid opening a million event socket connections, it makes sense to listen to a stream that retransmits events from all known repositories on the network: 255 255 256 - ![](./22-full.svg) 256 + ![The Leaflet app subscribes to updates from all user repositories by websocket.](./22-full.svg) 257 257 258 258 You can then filter down such a stream to just the events you're interested in, and then update your local database in response to the events your app cares about. 259 259 ··· 277 277 278 278 The pre-social web of "personalized sites" got data ownership and hosting independence, and linking right. Alice and Bob fully participate in the web: 279 279 280 - ![](./3-full.svg) 280 + ![Alice and Bob own their HTML.](./3-full.svg) 281 281 282 282 The closed social web innovated in scaling and in social aggregation features. Notifications, search, and feeds are non-negotiable in modern social products: 283 283 284 - ![](./6-full.svg) 284 + ![By putting stuff into a database, we can create aggregated screens.](./6-full.svg) 285 285 286 286 However, the closed social web has also excluded *us* from the web. *The web we create* is no longer meaningfully ours. We're just rows in somebody else's database. 287 287 288 - ![](./10-full.svg) 288 + ![The content is no longer ours. It lives inside closed boxes owned by social companies.](./10-full.svg) 289 289 290 290 Open social frees the web we're creating from somebody else's boxes. Our profiles, likes, follows, recipes, scrobbles, and other content meaningfully belongs to us: 291 291 292 - ![](./12-full.svg) 292 + ![Alice owns Alice's data. Bob owns Bob's data.](./12-full.svg) 293 293 294 294 The data no longer lives *inside* the products; the products *aggregate over* our data: 295 295 296 - ![](./22-full.svg) 296 + ![Products like Leaflet listen to events in Alice's and Bob's repos to construct their database.](./22-full.svg) 297 297 298 298 This blurs the boundaries between apps. Every open social app can use, remix, link to, and riff on data from every other open social app. 299 299 300 - ![](./19-full.svg) 300 + ![Records of different types flow into every app.](./19-full.svg) 301 301 302 302 The web we've created remains after the products we used to create it are gone. Developers can build new products to recontextualize it. No one can take it away. 303 303 304 - ![](./12-full.svg) 304 + ![Alice owns her data, Bob owns his data.](./12-full.svg) 305 305 306 306 As more products are built in the open social paradigm, [there's going to be a shift.](https://knotbin.leaflet.pub/3lx3uqveyj22f/) 307 307