A URL shortener service that uses ATProto to allow self hosting and ensuring the user owns their data

scope the running service instance to be for a single user

Signed-off-by: Will Andrews <did:plc:dadhhalkfcq3gucaq25hjqon>

+5 -5
README.md
··· 3 ### What is it? 4 It's a URL shorten service that uses ATProto so that the users creating the short links will always own the links they are creating and gives them the option to host the service themselves so that they can be sure that the links will always be available. 5 6 - Try it out [https://short.willdot.xyz] 7 - 8 ### Inspiration 9 This project was inspired by a conversation with someone about the classic engineering interview question about creating a URL shortning service. One of the downfalls of this is that you're relying on the service to always be there so that your links don't die. I think Google announced that they were shutting down their service which caused uproar because it meant that existing links embeded in websites and other places would cease to work. Thankfully they announced that they would allow existing links to work, just not allow new ones to be created. But that scare of links no longer working, made me think about how the ATProto architecture is perfect for allowing users to own their links and be sure that they will always be available without depending on another service. 10 11 ### How it works 12 13 - When a user logs in with their ATProto account, they will be able to create a new short URL. This action creates a record in their PDS containing the URL to redirect to and in the future could contain other peices of metadata. The key bit here is that the record in their PDS is created with an RKey, which in this application is a TID ((timestamp identifier)[https://atproto.com/specs/tid]) which is unique and that key becomes the "short URL" ID. Combine that with the host that created the ID and you get a short URL. The users DID, key and URL are then stored in a local database to the service. 14 15 As long as the service is running at that host, the short URL will be available to be redirected. When the short URL is hit, the service will take the key part of the URL, look it up in the database to find the real URL and then redirect to that URL. 16 ··· 20 21 ### Self hosting 22 23 - Another neat part of this architecture is that if a user wants to self host this, they can and the links they create will only work for their service. For example. 24 25 - If I create a short URL `https://my-short-service/a/abcd` on my service and then someone creates `https://a-different-short-service/a/1234` those links will only work with the domain that created them. I wouldn't be able to take the key `abcd` and use it with the `https://a-different-short-service` domain because that isn't the domain that created the link. This means that even though every service is consuming all of the short URL ATProto records, there will never be a case where someone could hijack an ID to redirect to somewhere else without the user that created and owning the link from doing so (because updating a record in a PDS requires the users auth).
··· 3 ### What is it? 4 It's a URL shorten service that uses ATProto so that the users creating the short links will always own the links they are creating and gives them the option to host the service themselves so that they can be sure that the links will always be available. 5 6 ### Inspiration 7 This project was inspired by a conversation with someone about the classic engineering interview question about creating a URL shortning service. One of the downfalls of this is that you're relying on the service to always be there so that your links don't die. I think Google announced that they were shutting down their service which caused uproar because it meant that existing links embeded in websites and other places would cease to work. Thankfully they announced that they would allow existing links to work, just not allow new ones to be created. But that scare of links no longer working, made me think about how the ATProto architecture is perfect for allowing users to own their links and be sure that they will always be available without depending on another service. 8 9 ### How it works 10 11 + When a user logs in with their ATProto account, they will be able to create a new short URL. This action creates a record in their PDS containing the URL to redirect to and in the future could contain other peices of metadata. The key bit here is that the record in their PDS is created with an RKey, which in this application is a TID ([timestamp identifier](https://atproto.com/specs/tid)) which is unique and that key becomes the "short URL" ID. Combine that with the host that created the ID and you get a short URL. The users DID, key and URL are then stored in a local database to the service. 12 13 As long as the service is running at that host, the short URL will be available to be redirected. When the short URL is hit, the service will take the key part of the URL, look it up in the database to find the real URL and then redirect to that URL. 14 ··· 18 19 ### Self hosting 20 21 + Another neat part of this architecture is that it allows the user to self host the service. Traditional short URL services turn out to be a lot of hassle. Read this really interesting [thread on Bluesky](https://bsky.app/profile/gbl08ma.com/post/3m2pft7a3dc23) about it from the creator of [tny.im](tny.im). Turns out hosting one for others to use isn't a great move these days, so by having one that you can host yourself and only you can use, could be a really handy tool. You create the short URLs and pass them out to people to use. 22 + 23 + The way the service works, is that whe you start it up, you configure it to be for your DID (ATProto identifier) so that only you can log into the service to create short URLs. Then when the Jetstream consumer runs, it only listens to events for your DID which means you only get your short URL records in the local database. 24 25 + Remember!!! All of the links you create are stored in your PDS which is public. So anyone that wants to see what links you've created and go to them, will be able to do that.
+5
auth_handlers.go
··· 25 return 26 } 27 28 next(w, r) 29 } 30 }
··· 25 return 26 } 27 28 + if did.String() != s.usersDID { 29 + http.Error(w, "not authorized", http.StatusUnauthorized) 30 + return 31 + } 32 + 33 next(w, r) 34 } 35 }
+10 -4
cmd/atshorter/main.go
··· 51 return 52 } 53 54 dbFilename := path.Join(dbMountPath, "database.db") 55 db, err := database.New(dbFilename) 56 if err != nil { ··· 94 }, 95 } 96 97 - server, err := atshorter.NewServer(host, port, db, oauthClient, httpClient) 98 if err != nil { 99 slog.Error("create new server", "error", err) 100 return ··· 112 _ = server.Stop(context.Background()) 113 }() 114 115 - go consumeLoop(ctx, db) 116 117 server.Run() 118 } 119 120 - func consumeLoop(ctx context.Context, db *database.DB) { 121 jsServerAddr := os.Getenv("JS_SERVER_ADDR") 122 if jsServerAddr == "" { 123 jsServerAddr = defaultServerAddr 124 } 125 126 - consumer := atshorter.NewConsumer(jsServerAddr, slog.Default(), db) 127 128 err := retry.Do(func() error { 129 err := consumer.Consume(ctx)
··· 51 return 52 } 53 54 + usersDID := os.Getenv("DID") 55 + if usersDID == "" { 56 + slog.Error("DID env not set") 57 + return 58 + } 59 + 60 dbFilename := path.Join(dbMountPath, "database.db") 61 db, err := database.New(dbFilename) 62 if err != nil { ··· 100 }, 101 } 102 103 + server, err := atshorter.NewServer(host, port, db, oauthClient, httpClient, usersDID) 104 if err != nil { 105 slog.Error("create new server", "error", err) 106 return ··· 118 _ = server.Stop(context.Background()) 119 }() 120 121 + go consumeLoop(ctx, db, usersDID) 122 123 server.Run() 124 } 125 126 + func consumeLoop(ctx context.Context, db *database.DB, did string) { 127 jsServerAddr := os.Getenv("JS_SERVER_ADDR") 128 if jsServerAddr == "" { 129 jsServerAddr = defaultServerAddr 130 } 131 132 + consumer := atshorter.NewConsumer(jsServerAddr, slog.Default(), db, did) 133 134 err := retry.Do(func() error { 135 err := consumer.Consume(ctx)
+2 -2
consumer.go
··· 19 logger *slog.Logger 20 } 21 22 - func NewConsumer(jsAddr string, logger *slog.Logger, store HandlerStore) *consumer { 23 cfg := client.DefaultClientConfig() 24 if jsAddr != "" { 25 cfg.WebsocketURL = jsAddr ··· 27 cfg.WantedCollections = []string{ 28 "com.atshorter.shorturl", 29 } 30 - cfg.WantedDids = []string{} // TODO: possibly when self hosting, limit this to just a select few? 31 32 return &consumer{ 33 cfg: cfg,
··· 19 logger *slog.Logger 20 } 21 22 + func NewConsumer(jsAddr string, logger *slog.Logger, store HandlerStore, did string) *consumer { 23 cfg := client.DefaultClientConfig() 24 if jsAddr != "" { 25 cfg.WebsocketURL = jsAddr ··· 27 cfg.WantedCollections = []string{ 28 "com.atshorter.shorturl", 29 } 30 + cfg.WantedDids = []string{did} 31 32 return &consumer{ 33 cfg: cfg,
+1
example.env
··· 4 DATABASE_PATH="./" 5 JS_SERVER_ADDR="set to a different Jetstream instance" 6 PORT="3002"
··· 4 DATABASE_PATH="./" 5 JS_SERVER_ADDR="set to a different Jetstream instance" 6 PORT="3002" 7 + DID="Enter your account DID here"
+3 -1
server.go
··· 29 30 type Server struct { 31 host string 32 httpserver *http.Server 33 sessionStore *sessions.CookieStore 34 templates []*template.Template ··· 44 //go:embed html 45 var htmlFolder embed.FS 46 47 - func NewServer(host string, port string, store Store, oauthClient *oauth.ClientApp, httpClient *http.Client) (*Server, error) { 48 sessionStore := sessions.NewCookieStore([]byte(os.Getenv("SESSION_KEY"))) 49 50 homeTemplate, err := template.ParseFS(htmlFolder, "html/home.html") ··· 64 65 srv := &Server{ 66 host: host, 67 oauthClient: oauthClient, 68 sessionStore: sessionStore, 69 templates: templates,
··· 29 30 type Server struct { 31 host string 32 + usersDID string 33 httpserver *http.Server 34 sessionStore *sessions.CookieStore 35 templates []*template.Template ··· 45 //go:embed html 46 var htmlFolder embed.FS 47 48 + func NewServer(host string, port string, store Store, oauthClient *oauth.ClientApp, httpClient *http.Client, usersDID string) (*Server, error) { 49 sessionStore := sessions.NewCookieStore([]byte(os.Getenv("SESSION_KEY"))) 50 51 homeTemplate, err := template.ParseFS(htmlFolder, "html/home.html") ··· 65 66 srv := &Server{ 67 host: host, 68 + usersDID: usersDID, 69 oauthClient: oauthClient, 70 sessionStore: sessionStore, 71 templates: templates,