mount an atproto PDS repository as a FUSE filesystem

Compare changes

Choose any two refs to compare.

Changed files
+159 -17
+3
allowed_signers
··· 1 + ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCivdIHm5jcKSMeVwdIIQj7bucOOBKnm2UD7TRRxOlo79UvdFDOg3W+hGYJfCVmDrqj7Ea1VMiytF/rbO45bcnsgpDce7A289NWBohsRMYXQ580/CHPepYhXTNL7mYOO/zTCcP6ABfK6d/iDl7J+6gx0NCAfyAL47fqqsIMEh4OvvghJvhhIMk/eQvEdXmqWXVqsX5d5heQCej8ii/1KsrVM/RYTcuIZuZ4nQyZEgIu+8A78dnKFos5Lg+tYUKKgjXyN3hbwpwgU4sSP9KgbrnyuVPBds2rulznzY0m/DRcJI0wyps5GoawX+VHpkP/jk/fhqblA9BRh8eO9fqSPMz7pOcQNdpZHpUoBqqKMQihkhq6/JVvgCRL7aka9umc9bcfh52or7fDSXTxNlFtrX8Cfw4Y5cjhCpGVXI8kbXgYGU9IHEB6DEEFwEKha6ND5Zr8JkXhhxpY4hFddQIg7/TxEmP0jpO7dixp0bGPjHauIJqe9iACK0ZCSNReVEdHPqc= steveklabnik@DESKTOP-VV370NK 2 + ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICJPYX06+qKr9IHWfkgCtHbExoBOOwS/+iAWbog9bAdk icy@wyndle 3 + ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICMmmaDFqSmbQLnPuTtg32wBdJs1xsituz3jrJBqlM1u anil@recoil.org
+156 -17
readme.txt
··· 1 - pdsfs 2 - ----- 1 + pdsfs - mount an atproto PDS repository as a FUSE filesystem. 3 2 4 - mount an atproto PDS repository as a fuse3 filesystem. 3 + a PDS repository[0] contains all data published by a user to 4 + the atmosphere[1]. it is exportable as a CAR 5 + (content-addressable archive) file. pdsfs is a tool that 6 + mounts this CAR file as a readonly-FUSE filesystem, allowing 7 + quick and easy exploration. 5 8 9 + to motivate the need for such a program, we could begin by 10 + mounting a repository: 6 11 7 - usage 8 - ----- 9 12 10 - cargo run -- [DIDs|handles ...] [-m mountpoint] 13 + λ pdsfs oppi.li 14 + mounted at "mnt" 15 + hit enter to unmount and exit... 16 + 17 + 18 + oppi.li is my handle in the atmosphere. the tool does some 19 + hardwork to determine the location of my PDS repository 20 + given my handle, but that is not important. lets have a look 21 + around: 22 + 23 + 24 + λ ls mnt/ 25 + did:plc:qfpnj4og54vl56wngdriaxug/ 26 + 27 + 28 + the did:plc:stuff is my DID. lets dig deeper: 29 + 30 + 31 + λ ls mnt/did\:plc\:qfpnj4og54vl56wngdriaxug/ 32 + app.bsky.actor.profile/ place.stream.chat.message/ 33 + app.bsky.actor.status/ place.stream.chat.profile/ 34 + app.bsky.feed.generator/ place.stream.key/ 35 + app.bsky.feed.like/ place.stream.livestream/ 36 + app.bsky.feed.post/ sh.tangled.actor.profile/ 37 + app.bsky.feed.repost/ sh.tangled.feed.reaction/ 38 + app.bsky.graph.block/ sh.tangled.feed.star/ 39 + app.bsky.graph.follow/ sh.tangled.graph.follow/ 40 + app.rocksky.album/ sh.tangled.knot/ 41 + app.rocksky.artist/ sh.tangled.knot.member/ 42 + . 43 + . 44 + . 45 + 46 + 47 + we have some data from the repository now. these are 48 + "collections". if i want to publish a post to bluesky, i 49 + would write content to the "app.bsky.feed.post" collection 50 + in my PDS. this will then be indexed by a bluesky appview 51 + (bsky.app or zeppelin.social to name a few) and show up 52 + under my profile there. 53 + 54 + pdsfs is kind enough to deserialize the data stored (as 55 + CBOR) in the PDS repository to JSON: 56 + 57 + 58 + λ cat sh.tangled.repo/3ljidbevrjh22 | jq 59 + { 60 + "$type": "sh.tangled.repo", 61 + "addedAt": "2025-03-03T16:04:13Z", 62 + "knot": "knot1.tangled.sh", 63 + "name": "hello-world", 64 + "owner": "did:plc:3danwc67lo7obz2fmdg6jxcr" 65 + } 66 + 67 + 68 + thanks pdsfs! 11 69 12 - example 13 - ------- 70 + i publish my music listening habits to my PDS to the 71 + "app.rocksky.scrobble" collection, because rocksky[4] 72 + recognizes and indexes this collection. i have wired up my 73 + personal navidrome instance to write data of this form into 74 + my PDS everytime i listen to a track. here are my top 75 + artists in order: 14 76 15 - $ cargo run -- oppi.li icyphox.sh 16 - ⠏ [00:00:00] using cached CAR file for...did:plc:qfpnj4og54vl56wngdriaxug 17 - ⠏ [00:00:00] using cached CAR file for...did:plc:hwevmowznbiukdf6uk5dwrrq 77 + 78 + λ cat app.rocksky.scrobble/* | jq -r '.artist' | sort | uniq -c | sort -nr 79 + 117 Thank You Scientist 80 + 45 FKJ 81 + 34 Covet 82 + 33 VOLA 83 + 23 Sam Cooke 84 + 22 Dark Tranquillity 85 + 21 Piero Piccioni 86 + 12 Bloodywood 87 + 11 Frank Sinatra 88 + 10 Dream Theater 89 + 90 + 91 + it is true, i love sam cooke. 92 + 93 + pdsfs allows mounting multiple repositories at a time. allow 94 + me to introduce my friends: 95 + 96 + 97 + λ pdsfs icyphox.sh anil.recoil.org steveklabnik.com tangled.sh 98 + using cached CAR file for...did:plc:hwevmowznbiukdf6uk5dwrrq 99 + using cached CAR file for...did:plc:nhyitepp3u4u6fcfboegzcjw 100 + download complete for...did:plc:3danwc67lo7obz2fmdg6jxcr 101 + download complete for...did:plc:wshs7t2adsemcrrd4snkeqli 18 102 mounted at "mnt" 19 103 hit enter to unmount and exit... 20 104 21 105 22 - $ cat foo/**/sh.tangled.publicKey/* | jq -r '.key' 23 - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICJPYX06+qKr9IHWfkgCtHbExoBOOwS/+iAWbog9bAdk icy@wyndle 24 - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMj1Dn9YuFo2BNr993ymBa6nzyyIKAURIqMbUtfI8+4X op@mantis 25 - . 26 - . 27 - . 106 + # -- in a separate shell -- 107 + 108 + 109 + λ cat ./mnt/*/app.bsky.actor.profile/* \ 110 + | jq -r '"\(.displayName)\n\(.description)\n---"' \ 111 + | sed '/^$/d' 112 + Steve Klabnik 113 + #rustlang, #jj-vcs, atproto, shitposts, urbanism. I 114 + contain multitudes. Working on #ruelang but just for 115 + fun. Currently in Austin, TX, but from Pittsburgh. 116 + Previously in Bushwick, the Mission, LA. 117 + --- 118 + Anirudh Oppiliappan 119 + building @tangled.sh — code collaboration platform built 120 + on atproto helsinki, finland · https://anirudh.fi · 121 + (somewhat) effective altruist 122 + --- 123 + Anil Madhavapeddy 124 + Professor of Planetary Computing at the University of 125 + Cambridge @cst.cam.ac.uk, where I co-lead the 126 + @eeg.cl.cam.ac.uk, and am also to found at 127 + @conservation.cam.ac.uk. Homepage at 128 + https://anil.recoil.org 129 + --- 130 + Tangled 131 + https://tangled.sh is a git collaboration platform built 132 + on atproto. Social coding, but for real this time! 133 + Discord: chat.tangled.sh IRC: #tangled @ libera.chat 134 + Built by @oppi.li & @icyphox.sh 135 + --- 136 + 137 + 138 + all my friends use tangled.sh, which requires them to 139 + publish their ssh public key to their PDii (PDSes?). perhaps 140 + i would like to add their keys to my allowed_signers file to 141 + verify their commit signatures: 142 + 143 + 144 + λ for dir in ./*/sh.tangled.publicKey; 145 + do cat $dir/$(ls -r $dir | head -n1) | jq -r '.key'; 146 + done | tee allowed_signers 147 + ssh-rsa AAAAB3NzaC1yc2EAAA...dHPqc= steveklabnik@DESKTOP-VV370NK 148 + ssh-ed25519 AAAAC3NzaC1lZD...g9bAdk icy@wyndle 149 + ssh-ed25519 AAAAC3NzaC1lZD...BqlM1u anil@recoil.org 150 + 151 + 152 + 153 + --- 154 + 28 155 156 + FUSE is quite liberating in that it allows you to represent 157 + anything as a filesystem. when applications like ls and cat 158 + are executed, the system calls to open and read are rerouted 159 + to your custom fs implementation (pdsfs in this case). the 160 + custom fs implementation is free to as it pleases, in fact 161 + the first iteration of pdsfs accessed the network for each 162 + open/read call to fetch live data from PDii. 29 163 30 164 165 + [0]: https://atproto.com/guides/data-repos 166 + [1]: https://atproto.com 167 + [2]: https://docs.bsky.app/docs/api/com-atproto-sync-get-repo 168 + [3]: https://tangled.sh/keys/oppi.li 169 + [4]: https://rocksky.app