···7788As well, the Bluesky PDS is not currently accepting incoming migrations (it will in the future). Therefore this is currently a one-way street. If you migrate off of `bsky.social`, _you will not be able to return_. However, you will be able to migrate between other PDSs.
991010-
1010+...
1111+1212+
11131214Account Migration occurs in 4 main steps:
1315- Creating an account on the new PDS
···67696870## Example Code
69717070-The below Typescript code gives an example of how this account migration flow may function. Please note that it is for documentation purposes only and can not be run exactly as is as there is an out-of-band step where you need to get a confirmation token from your email.
7272+The below code gives an example of how this account migration flow may function. Please note that it is for documentation purposes only and can not be run exactly as is as there is an out-of-band step where you need to get a confirmation token from your email.
71737274It does also not handle some of the more advanced steps such as verifying a full import, looking for missing blobs, adding your own recovery key, or validating the PLC operation itself.
7375
+4-8
Dockerfile
···11-FROM node:20.11-alpine3.18 as build
22-33-RUN npm install -g pnpm
11+FROM node:18-alpine as build
4253# Move files into the image and install
64WORKDIR /app
75COPY ./service ./
88-RUN pnpm install --production --frozen-lockfile > /dev/null
66+RUN yarn install --production --frozen-lockfile > /dev/null
97108# Uses assets from build stage to reduce build size
1111-FROM node:20.11-alpine3.18
99+FROM node:18-alpine
12101311RUN apk add --update dumb-init
1412···2119EXPOSE 3000
2220ENV PDS_PORT=3000
2321ENV NODE_ENV=production
2424-# potential perf issues w/ io_uring on this version of node
2525-ENV UV_USE_IO_URING=0
26222723CMD ["node", "--enable-source-maps", "index.js"]
28242925LABEL org.opencontainers.image.source=https://github.com/bluesky-social/pds
3030-LABEL org.opencontainers.image.description="AT Protocol PDS"
2626+LABEL org.opencontainers.image.description="ATP Personal Data Server (PDS)"
3127LABEL org.opencontainers.image.licenses=MIT
+279-60
README.md
···4455## Table of Contents
6677-<!-- markdown-toc -i README.md -->
88-99-<!-- toc -->
1010-1111-- [FAQ](#faq)
77+* [FAQ](#faq)
128 * [What is Bluesky?](#what-is-bluesky)
139 * [What is AT Protocol?](#what-is-at-protocol)
1010+ * [How can developers get invite codes?](#how-can-developers-get-invite-codes)
1411 * [Where is the code?](#where-is-the-code)
1512 * [What is the current status of federation?](#what-is-the-current-status-of-federation)
1613 * [What should I know about running a PDS in the developer sandbox?](#what-should-i-know-about-running-a-pds-in-the-developer-sandbox)
1717-- [Self-hosting PDS](#self-hosting-pds)
1818- * [Preparation for self-hosting PDS](#preparation-for-self-hosting-pds)
1414+* [Self\-hosting PDS](#self-hosting-pds)
1515+ * [Preparation for self\-hosting PDS](#preparation-for-self-hosting-pds)
1916 * [Open your cloud firewall for HTTP and HTTPS](#open-your-cloud-firewall-for-http-and-https)
2017 * [Configure DNS for your domain](#configure-dns-for-your-domain)
2118 * [Check that DNS is working as expected](#check-that-dns-is-working-as-expected)
2222- * [Installer on Ubuntu 20.04/22.04 and Debian 11/12](#installer-on-ubuntu-20042204-and-debian-1112)
2323- * [Verifying that your PDS is online and accessible](#verifying-that-your-pds-is-online-and-accessible)
2424- * [Creating an account using pdsadmin](#creating-an-account-using-pdsadmin)
2525- * [Creating an account using an invite code](#creating-an-account-using-an-invite-code)
2626- * [Using the Bluesky app with your PDS](#using-the-bluesky-app-with-your-pds)
2727- * [Updating your PDS](#updating-your-pds)
1919+ * [Automatic install on Ubuntu 20\.04/22\.04 or Debian 11/12](#automatic-install-on-ubuntu-20042204-or-debian-1112)
2020+ * [Installing manually on Ubuntu 22\.04](#installing-manually-on-ubuntu-2204)
2121+ * [Open ports on your Linux firewall](#open-ports-on-your-linux-firewall)
2222+ * [Install Docker](#install-docker)
2323+ * [Uninstall old versions](#uninstall-old-versions)
2424+ * [Set up the repository](#set-up-the-repository)
2525+ * [Install Docker Engine](#install-docker-engine)
2626+ * [Verify Docker Engine installation](#verify-docker-engine-installation)
2727+ * [Set up the PDS directory](#set-up-the-pds-directory)
2828+ * [Create the Caddyfile](#create-the-caddyfile)
2929+ * [Create the PDS env configuration file](#create-the-pds-env-configuration-file)
3030+ * [Start the PDS containers](#start-the-pds-containers)
3131+ * [Download the Docker compose file](#download-the-docker-compose-file)
3232+ * [Create the systemd service](#create-the-systemd-service)
3333+ * [Start the service](#start-the-service)
3434+ * [Verify your PDS is online](#verify-your-pds-is-online)
3535+ * [Obtain your PDS admin password](#obtain-your-pds-admin-password)
3636+ * [Generate an invite code for your PDS](#generate-an-invite-code-for-your-pds)
3737+ * [Connecting to your server](#connecting-to-your-server)
3838+ * [Manually updating your PDS](#manually-updating-your-pds)
3939+* [PDS environment variables](#pds-environment-variables)
28402929-<!-- tocstop -->
30413142## FAQ
3243···34453546Bluesky is a social media application built on AT Protocol.
36473737-Please visit the [Bluesky website](https://bsky.social/) for more information.
4848+Please visit the [Bluesky website](https://bsky.app/) for more information.
38493950### What is AT Protocol?
4051···42534354Please visit the [AT Protocol docs](https://atproto.com/guides/overview) for additional information.
44555656+### How can developers get invite codes?
5757+5858+There is no invite required to join the sandbox network. Simply set up your own PDS and generate your own invite codes to create accounts. If you desire an account on the production network (on the official Bluesky PDS) please check out the [Bluesky Developer Waitlist](https://docs.google.com/forms/d/e/1FAIpQLSfCuguykw3HaPxIZMJQKRu8_-vsRew90NALVTDOjCSPDmsGNg/viewform) which prioritizes access for developers wanting to build software on atproto.
5959+4560### Where is the code?
46614747-* [TypeScript code](https://github.com/bluesky-social/atproto)
4848-* [Go code](https://github.com/bluesky-social/indigo)
6262+* [Canonical TypeScript code](https://github.com/bluesky-social/atproto)
6363+* [Experimental Go code](https://github.com/bluesky-social/indigo)
49645065### What is the current status of federation?
51665252-As of Feb, 2024, the AT Protocol data service (PDS) is now open to federation for self-hosters!
5353-5454-โ Federated domain handles (e.g. `@nytimes.com`)
5555-5656-โ Federated feed generators (custom algorithms)
5757-5858-โ Federated relays (event firehose)
5959-6060-โ Federated app views (API service)
6161-6262-โ Federated data for self-hosters (PDS hosting)
6363-6464-๐ฉ Federated moderation (labeling) (coming soon)
6565-6666-๐ฉ Federated data for large service providers (coming soon)
6767+We do not currently support PDS federation on the production network but it is now possible to federate in the developer sandbox.
67686869### What should I know about running a PDS in the developer sandbox?
6969-7070-Developers may now run self-hosted PDS hosts on the production network!
7171-7272-Though it is still recommended to run experiments in the developer sandbox network.
73707471Read the [SANDBOX.md](https://github.com/bluesky-social/pds/blob/main/SANDBOX.md) for an overview of the sandbox network.
7572···137134138135These should all return your server's public IP.
139136140140-### Installer on Ubuntu 20.04/22.04 and Debian 11/12
137137+### Automatic install on Ubuntu 20.04/22.04 or Debian 11/12
141138142142-On your server via ssh, download the installer script using wget:
139139+On your server via ssh, run the installer script:
143140144141```bash
145142wget https://raw.githubusercontent.com/bluesky-social/pds/main/installer.sh
146143```
147144148148-or download it using curl:
145145+```bash
146146+sudo bash installer.sh
147147+```
148148+149149+### Installing manually on Ubuntu 22.04
150150+151151+#### Open ports on your Linux firewall
152152+153153+If your server is running a Linux firewall managed with `ufw`, you will need to open these ports:
154154+155155+```bash
156156+$ sudo ufw allow 80/tcp
157157+$ sudo ufw allow 443/tcp
158158+```
159159+160160+#### Install Docker
161161+162162+On your server, install Docker CE (Community Edition), using the the following instructions. For other operating systems you may reference the [official Docker install guides](https://docs.docker.com/engine/install/).
163163+164164+**Note:** All of the following commands should be run on your server via ssh.
165165+166166+##### Uninstall old versions
167167+168168+```bash
169169+sudo apt-get remove docker docker-engine docker.io containerd runc
170170+```
171171+172172+##### Set up the repository
173173+174174+```bash
175175+sudo apt-get update
176176+sudo apt-get install \
177177+ ca-certificates \
178178+ curl \
179179+ gnupg
180180+```
181181+182182+```bash
183183+sudo install -m 0755 -d /etc/apt/keyrings
184184+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
185185+sudo chmod a+r /etc/apt/keyrings/docker.gpg
186186+```
187187+188188+```bash
189189+echo \
190190+ "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
191191+ "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
192192+ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
193193+```
194194+195195+##### Install Docker Engine
196196+197197+```bash
198198+sudo apt-get update
199199+```
200200+201201+```bash
202202+sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
203203+```
204204+205205+##### Verify Docker Engine installation
206206+207207+```bash
208208+sudo docker run hello-world
209209+```
210210+211211+#### Set up the PDS directory
212212+213213+```bash
214214+sudo mkdir /pds
215215+sudo mkdir --parents /pds/caddy/data
216216+sudo mkdir --parents /pds/caddy/etc/caddy
217217+```
218218+219219+#### Create the Caddyfile
220220+221221+Be sure to replace `example.com` with your own domain.
222222+223223+```bash
224224+cat <<CADDYFILE | sudo tee /pds/caddy/etc/caddy/Caddyfile
225225+{
226226+ email you@example.com
227227+}
228228+229229+*.example.com, example.com {
230230+ tls {
231231+ on_demand
232232+ }
233233+ reverse_proxy http://localhost:3000
234234+}
235235+CADDYFILE
236236+```
237237+238238+#### Create the PDS env configuration file
239239+240240+You should fill in the first 5 values, but leave the rest untouched unless you have good reason to change it.
241241+242242+See the PDS environment variables section at the end of this README for explanations of each value
243243+244244+Your PDS will need two secp256k1 private keys provided as hex strings. You can securely generate these keys using `openssl` with the following command:
245245+246246+**Note:**
247247+* Replace `example.com` with your domain name.
248248+249249+```bash
250250+PDS_HOSTNAME="example.com"
251251+PDS_JWT_SECRET="$(openssl rand --hex 16)"
252252+PDS_ADMIN_PASSWORD="$(openssl rand --hex 16)"
253253+PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX="$(openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32)"
254254+PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX="$(openssl ecparam --name secp256k1 --genkey --noout --outform DER | tail --bytes=+8 | head --bytes=32 | xxd --plain --cols 32)"
255255+256256+cat <<PDS_CONFIG | sudo tee /pds/pds.env
257257+PDS_HOSTNAME=${PDS_HOSTNAME}
258258+PDS_JWT_SECRET=${PDS_JWT_SECRET}
259259+PDS_ADMIN_PASSWORD=${PDS_ADMIN_PASSWORD}
260260+PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX=${PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX}
261261+PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX=${PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX}
262262+PDS_DB_SQLITE_LOCATION=/pds/pds.sqlite
263263+PDS_BLOBSTORE_DISK_LOCATION=/pds/blocks
264264+PDS_DID_PLC_URL=https://plc.bsky-sandbox.dev
265265+PDS_BSKY_APP_VIEW_URL=https://api.bsky-sandbox.dev
266266+PDS_BSKY_APP_VIEW_DID=did:web:api.bsky-sandbox.dev
267267+PDS_CRAWLERS=https://bgs.bsky-sandbox.dev
268268+PDS_CONFIG
269269+```
270270+271271+#### Start the PDS containers
272272+273273+##### Download the Docker compose file
274274+275275+Download the `compose.yaml` to run your PDS, which includes the following containers:
276276+277277+* `pds` Node PDS server running on http://localhost:3000
278278+* `caddy` HTTP reverse proxy handling TLS and proxying requests to the PDS server
279279+* `watchtower` Daemon responsible for auto-updating containers to keep the server secure and federating
280280+281281+```bash
282282+curl https://raw.githubusercontent.com/bluesky-social/pds/main/compose.yaml | sudo tee /pds/compose.yaml
283283+```
284284+285285+##### Create the systemd service
149286150287```bash
151151-curl https://raw.githubusercontent.com/bluesky-social/pds/main/installer.sh >installer.sh
288288+ cat <<SYSTEMD_UNIT_FILE >/etc/systemd/system/pds.service
289289+[Unit]
290290+Description=Bluesky PDS Service
291291+Documentation=https://github.com/bluesky-social/pds
292292+Requires=docker.service
293293+After=docker.service
294294+295295+[Service]
296296+Type=oneshot
297297+RemainAfterExit=yes
298298+WorkingDirectory=/pds
299299+ExecStart=/usr/bin/docker compose --file /pds/compose.yaml up --detach
300300+ExecStop=/usr/bin/docker compose --file /pds/compose.yaml down
301301+302302+[Install]
303303+WantedBy=default.target
304304+SYSTEMD_UNIT_FILE
152305```
153306154154-And then run the installer using bash:
307307+##### Start the service
155308309309+**Reload the systemd daemon to create the new service:**
156310```bash
157157-sudo bash installer.sh
311311+sudo systemctl daemon-reload
312312+```
313313+314314+**Enable the systemd service:**
315315+```bash
316316+sudo systemctl enable pds
158317```
159318160160-### Verifying that your PDS is online and accessible
319319+**Start the pds systemd service:**
320320+```bash
321321+sudo systemctl start pds
322322+```
161323162162-You can check if your server is online and healthy by requesting the healthcheck endpoint.
324324+**Ensure that containers are running**
163325164164-You can visit `https://example.com/xrpc/_health` in your browser. You should see a JSON response with a version.
326326+There should be a caddy, pds, and watchtower container running.
165327166166-For example:
328328+```bash
329329+sudo systemctl status pds
330330+```
167331332332+```bash
333333+sudo docker ps
168334```
335335+336336+### Verify your PDS is online
337337+338338+You can check if your server is online and healthy by requesting the healthcheck endpoint.
339339+340340+```bash
341341+curl https://example.com/xrpc/_health
169342{"version":"0.2.2-beta.2"}
170343```
171344172172-### Creating an account using pdsadmin
345345+### Obtain your PDS admin password
346346+347347+Your PDS admin password should be in your `pds.env` file if you used the installer script.
173348174174-Using ssh on your server, use `pdsadmin` to create an account if you haven't already.
349349+**For example:**
175350176351```bash
177177-sudo pdsadmin account create
352352+$ source /pds/pds.env
353353+$ echo $PDS_ADMIN_PASSWORD
354354+a7b5970b6a5077bb41fc68a26d30adda
178355```
356356+### Generate an invite code for your PDS
179357180180-### Creating an account using an invite code
358358+By default, your PDS will require an invite code to create an account.
181359182182-Using ssh on your server, use `pdsadmin` to create an invite code.
360360+You can generate a new invite code with the following command:
183361184362```bash
185185-sudo pdsadmin create-invite-code
363363+PDS_HOSTNAME="example.com"
364364+PDS_ADMIN_PASSWORD="<YOUR PDS ADMIN PASSWORD>"
365365+366366+curl --silent \
367367+ --show-error \
368368+ --request POST \
369369+ --user "admin:${PDS_ADMIN_PASSWORD}" \
370370+ --header "Content-Type: application/json" \
371371+ --data '{"useCount": 1}' \
372372+ https://${PDS_HOSTNAME}/xrpc/com.atproto.server.createInviteCode
186373```
187374188188-When creating an account using the app, enter this invite code.
375375+**Note:** the `useCount` field specifies how many times an invite code can be used
189376190190-### Using the Bluesky app with your PDS
377377+### Connecting to your server
191378192192-You can use the Bluesky app to connect to your PDS.
379379+You can use the Bluesky app to connect to your server to create an account.
1933801943811. Get the Bluesky app
195195- * [Bluesky for Web](https://bsky.app/)
382382+ * [Bluesky for Web (sandbox)](https://app.bsky-sandbox.dev/)
196383 * [Bluesky for iPhone](https://apps.apple.com/us/app/bluesky-social/id6444370199)
197384 * [Bluesky for Android](https://play.google.com/store/apps/details?id=xyz.blueskyweb.app)
1983851. Enter the URL of your PDS (e.g. `https://example.com/`)
386386+1. Create an account using the generated invite code
387387+1. Create a post
388388+389389+_Note: because we use on-the-fly TLS certs, it may take 10-30s for your handle to be accessible. If you aren't seeing your first post/profile, wait 30s and try to make another post._
199390200200-_Note: because the subdomain TLS certificate is created on-demand, it may take 10-30s for your handle to be accessible. If you aren't seeing your first post/profile, wait 30s and try to make another post._
391391+Checkout [SANDBOX.md](./SANDBOX.md) for an overview of participating in the sandbox network.
392392+393393+### Manually updating your PDS
201394202202-### Updating your PDS
395395+If you use use Docker `compose.yaml` file in this repo, your PDS will automatically update nightly. To manually update to the latest version use the following commands.
203396204204-It is recommended that you keep your PDS up to date with new versions, otherwise things may break. You can use the `pdsadmin` tool to update your PDS.
397397+**Pull the latest PDS container image:**
398398+```bash
399399+sudo docker pull ghcr.io/bluesky-social/pds:latest
400400+```
205401402402+**Restart PDS with the new container image:**
206403```bash
207207-sudo pdsadmin update
404404+sudo systemctl restart pds
208405```
406406+407407+## PDS environment variables
408408+409409+You will need to customize various settings configured through the PDS environment variables. See the below table to find the variables you'll need to set.
410410+411411+| Environment Variable | Value | Should update? | Notes |
412412+| ----------------------------------------- | ---------------------------- | -------------- | --------------------------------------------------------------------------- |
413413+| PDS_HOSTNAME | example.com | โ | Public domain you intend to deploy your service at |
414414+| PDS_JWT_SECRET | jwt-secret | โ | Use a secure high-entropy string that is 32 characters in length |
415415+| PDS_ADMIN_PASSWORD | admin-pass | โ | Use a secure high-entropy string that is 32 characters in length |
416416+| PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX | 3ee68... | โ | See above Generate Keys section - once set, do not change |
417417+| PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX | e049f... | โ | See above Generate Keys section - once set, do not change |
418418+| PDS_DB_SQLITE_LOCATION | /pds/pds.sqlite | โ | Or use `PDS_DB_POSTGRES_URL` depending on which database you intend to use |
419419+| PDS_BLOBSTORE_DISK_LOCATION | /pds/blocks | โ | Only update if you update the mounted volume for your docker image as well |
420420+| PDS_DID_PLC_URL | https://plc.bsky-sandbox.dev | โ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
421421+| PDS_BSKY_APP_VIEW_URL | https://api.bsky-sandbox.dev | โ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
422422+| PDS_BSKY_APP_VIEW_DID | did:web:api.bsky-sandbox.dev | โ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
423423+| PDS_CRAWLERS | https://bgs.bsky-sandbox.dev | โ | Do not adjust if you intend to federate with the Bluesky federation sandbox |
424424+425425+There are additional environment variables that can be tweaked depending on how you're running your service. For instance, storing blobs in AWS S3, keys in AWS KMS, or setting up an email service.
426426+427427+Feel free to explore those [Here](https://github.com/bluesky-social/atproto/blob/simplify-pds/packages/pds/src/config/env.ts). However, we will not be providing support for more advanced configurations.
+3-29
SANDBOX.md
···10101111Given that this is a testing environment, we will be defederating from any instances that do not abide by these guidelines, or that cause unnecessary trouble, and will not be providing specific justifications for these decisions.
12121313-<!-- markdown-toc -i SANDBOX.md -->
1414-1515-<!-- toc -->
1616-1717-- [Guidelines that must be followed](#guidelines-that-must-be-followed)
1818- * [Post responsibly](#post-responsibly)
1919- * [Keep the emphasis on testing](#keep-the-emphasis-on-testing)
2020- * [Do limit account creation](#do-limit-account-creation)
2121- * [Donโt expect persistence or uptime](#dont-expect-persistence-or-uptime)
2222- * [Don't advertise your service as being "Bluesky"](#dont-advertise-your-service-as-being-bluesky)
2323- * [Do not mirror sandbox did:plcs to production](#do-not-mirror-sandbox-didplcs-to-production)
2424- * [Status and Wipes](#status-and-wipes)
2525- + [๐ Beware of dragons!](#%F0%9F%90%89-beware-of-dragons)
2626- + [Routine wipes](#routine-wipes)
2727-- [Getting started](#getting-started)
2828- * [Auto-updates](#auto-updates)
2929- * [Odds & Ends & Warnings & Reminders](#odds--ends--warnings--reminders)
3030-- [Learn more about atproto federation](#learn-more-about-atproto-federation)
3131- * [Network Services](#network-services)
3232- + [PLC](#plc)
3333- + [BGS](#bgs)
3434- + [Bluesky App View](#bluesky-app-view)
3535-- [The PDS](#the-pds)
3636-3737-<!-- tocstop -->
3838-3913# Guidelines that must be followed
40144115Using the sandbox environment means you agree to adhere to our Guidelines. Please read the following carefully:
···44184519The sandbox environment is intended to test infrastructure, but user content may be created as part of this testing process. Content generation can be automated or manual.
46204747-Do not post content that requires active moderation or violates the [Bluesky Community Guidelines](https://bsky.social/about/support/community-guidelines).
2121+Do not post content that requires active moderation or violates the [Bluesky Community Guidelines](https://blueskyweb.xyz/support/community-guidelines).
48224923## Keep the emphasis on testing
5024···1199312094# Learn more about atproto federation
12195122122-Check out the [high-level view of federation](https://bsky.social/about/blog/5-5-2023-federation-architecture).
9696+Check out the [high-level view of federation](https://blueskyweb.xyz/blog/5-5-2023-federation-architecture).
1239712498Dive deeper with the [atproto docs](https://atproto.com/docs).
12599···165139166140Weโre not actually running a Bluesky PDS in sandbox. You might see Bluesky team members' accounts in the sandbox environment, but those are self-hosted too.
167141168168-The PDS that youโll be running is much of the same code that is running on the Bluesky production PDS. Notably, all of the in-pds-appview code has been torn out. You can see the actual PDS code that youโre running on the [atproto/simplify-pds](https://github.com/bluesky-social/atproto/pull/1198) branch.
142142+The PDS that youโll be running is much of the same code that is running on the Bluesky production PDS. Notably, all of the in-pds-appview code has been torn out. You can see the actual PDS code that youโre running on the [atproto/simplify-pds](https://github.com/bluesky-social/atproto/pull/1198) branch.
···1818# The Docker compose file.
1919COMPOSE_URL="https://raw.githubusercontent.com/bluesky-social/pds/main/compose.yaml"
20202121-# The pdsadmin script.
2222-PDSADMIN_URL="https://raw.githubusercontent.com/bluesky-social/pds/main/pdsadmin.sh"
2323-2421# System dependencies.
2522REQUIRED_SYSTEM_PACKAGES="
2623 ca-certificates
2724 curl
2825 gnupg
2929- jq
3026 lsb-release
3127 openssl
3232- sqlite3
3328 xxd
3429"
3530# Docker packages.
3631REQUIRED_DOCKER_PACKAGES="
3737- containerd.io
3832 docker-ce
3933 docker-ce-cli
4034 docker-compose-plugin
3535+ containerd.io
4136"
42374338PUBLIC_IP=""
···5045PDS_DATADIR="${1:-/pds}"
5146PDS_HOSTNAME="${2:-}"
5247PDS_ADMIN_EMAIL="${3:-}"
5353-PDS_DID_PLC_URL="https://plc.directory"
5454-PDS_BSKY_APP_VIEW_URL="https://api.bsky.app"
5555-PDS_BSKY_APP_VIEW_DID="did:web:api.bsky.app"
5656-PDS_REPORT_SERVICE_URL="https://mod.bsky.app"
5757-PDS_REPORT_SERVICE_DID="did:plc:ar7c4by46qjdydhdevvrndac"
5858-PDS_CRAWLERS="https://bsky.network"
4848+PDS_DID_PLC_URL="https://plc.bsky-sandbox.dev"
4949+PDS_BSKY_APP_VIEW_ENDPOINT="https://api.bsky-sandbox.dev"
5050+PDS_BSKY_APP_VIEW_DID="did:web:api.bsky-sandbox.dev"
5151+PDS_CRAWLERS="https://bgs.bsky-sandbox.dev"
59526053function usage {
6154 local error="${1}"
···9386 elif [[ "${DISTRIB_CODENAME}" == "jammy" ]]; then
9487 SUPPORTED_OS="true"
9588 echo "* Detected supported distribution Ubuntu 22.04 LTS"
9696- elif [[ "${DISTRIB_CODENAME}" == "mantic" ]]; then
9797- SUPPORTED_OS="true"
9898- echo "* Detected supported distribution Ubuntu 23.10 LTS"
9989 fi
10090 elif [[ "${DISTRIB_ID}" == "debian" ]]; then
10191 if [[ "${DISTRIB_CODENAME}" == "bullseye" ]]; then
···112102 exit 1
113103 fi
114104115115- # Enforce that the data directory is /pds since we're assuming it for now.
116116- # Later we can make this actually configurable.
117117- if [[ "${PDS_DATADIR}" != "/pds" ]]; then
118118- usage "The data directory must be /pds. Exiting..."
119119- fi
120120-121105 # Check if PDS is already installed.
122106 if [[ -e "${PDS_DATADIR}/pds.sqlite" ]]; then
123107 echo
···140124 echo "For assistance, check https://github.com/bluesky-social/pds"
141125 exit 1
142126 fi
127127+143128144129 #
145130 # Attempt to determine server's public IP.
···181166182167 From your DNS provider's control panel, create the required
183168 DNS record with the value of your server's public IP address.
184184-169169+185170 + Any DNS name that can be resolved on the public internet will work.
186171 + Replace example.com below with any valid domain name you control.
187172 + A TTL of 600 seconds (10 minutes) is recommended.
188188-173173+189174 Example DNS record:
190190-175175+191176 NAME TYPE VALUE
192177 ---- ---- -----
193178 example.com A ${PUBLIC_IP:-Server public IP}
···228213 usage "No admin email specified"
229214 fi
230215216216+231217 #
232218 # Install system packages.
233219 #
···241227 sleep 2
242228 done
243229 fi
244244-230230+245231 apt-get update
246232 apt-get install --yes ${REQUIRED_SYSTEM_PACKAGES}
247233···309295{
310296 email ${PDS_ADMIN_EMAIL}
311297 on_demand_tls {
312312- ask http://localhost:3000/tls-check
298298+ ask http://localhost:3000
313299 }
314300}
315301···330316PDS_HOSTNAME=${PDS_HOSTNAME}
331317PDS_JWT_SECRET=$(eval "${GENERATE_SECURE_SECRET_CMD}")
332318PDS_ADMIN_PASSWORD=${PDS_ADMIN_PASSWORD}
319319+PDS_REPO_SIGNING_KEY_K256_PRIVATE_KEY_HEX=$(eval "${GENERATE_K256_PRIVATE_KEY_CMD}")
333320PDS_PLC_ROTATION_KEY_K256_PRIVATE_KEY_HEX=$(eval "${GENERATE_K256_PRIVATE_KEY_CMD}")
334334-PDS_DATA_DIRECTORY=${PDS_DATADIR}
321321+PDS_DB_SQLITE_LOCATION=${PDS_DATADIR}/pds.sqlite
335322PDS_BLOBSTORE_DISK_LOCATION=${PDS_DATADIR}/blocks
336323PDS_DID_PLC_URL=${PDS_DID_PLC_URL}
337337-PDS_BSKY_APP_VIEW_URL=${PDS_BSKY_APP_VIEW_URL}
324324+PDS_BSKY_APP_VIEW_ENDPOINT=${PDS_BSKY_APP_VIEW_ENDPOINT}
338325PDS_BSKY_APP_VIEW_DID=${PDS_BSKY_APP_VIEW_DID}
339339-PDS_REPORT_SERVICE_URL=${PDS_REPORT_SERVICE_URL}
340340-PDS_REPORT_SERVICE_DID=${PDS_REPORT_SERVICE_DID}
341326PDS_CRAWLERS=${PDS_CRAWLERS}
342342-LOG_ENABLED=true
343327PDS_CONFIG
344328345329 #
···394378 fi
395379 fi
396380397397- #
398398- # Download and install pdadmin.
399399- #
400400- echo "* Downloading pdsadmin"
401401- curl \
402402- --silent \
403403- --show-error \
404404- --fail \
405405- --output "/usr/local/bin/pdsadmin" \
406406- "${PDSADMIN_URL}"
407407- chmod +x /usr/local/bin/pdsadmin
408408-409381 cat <<INSTALLER_MESSAGE
410382========================================================================
411411-PDS installation successful!
383383+PDS installation successful!
412384------------------------------------------------------------------------
413385414386Check service status : sudo systemctl status pds
415387Watch service logs : sudo docker logs -f pds
416388Backup service data : ${PDS_DATADIR}
417417-PDS Admin command : pdsadmin
418389419390Required Firewall Ports
420391------------------------------------------------------------------------
···425396426397Required DNS entries
427398------------------------------------------------------------------------
428428-Name Type Value
399399+Name Type Value
429400------- --------- ---------------
430430-${PDS_HOSTNAME} A ${PUBLIC_IP}
431431-*.${PDS_HOSTNAME} A ${PUBLIC_IP}
401401+${PDS_HOSTNAME} A ${PUBLIC_IP}
402402+*.${PDS_HOSTNAME} A ${PUBLIC_IP}
432403433404Detected public IP of this server: ${PUBLIC_IP}
434405435435-To see pdsadmin commands, run "pdsadmin help"
406406+# To create an invite code, run the following command:
407407+408408+curl --silent \\
409409+ --show-error \\
410410+ --request POST \\
411411+ --user "admin:${PDS_ADMIN_PASSWORD}" \\
412412+ --header "Content-Type: application/json" \\
413413+ --data '{"useCount": 1}' \\
414414+ https://${PDS_HOSTNAME}/xrpc/com.atproto.server.createInviteCode
436415437416========================================================================
438417INSTALLER_MESSAGE
439439-440440- CREATE_ACCOUNT_PROMPT=""
441441- read -p "Create a PDS user account? (y/N): " CREATE_ACCOUNT_PROMPT
442442-443443- if [[ "${CREATE_ACCOUNT_PROMPT}" =~ ^[Yy] ]]; then
444444- pdsadmin account create
445445- fi
446446-447418}
448419449420# Run main function.
-234
pdsadmin/account.sh
···11-#!/bin/bash
22-set -o errexit
33-set -o nounset
44-set -o pipefail
55-66-PDS_ENV_FILE="/pds/pds.env"
77-source "${PDS_ENV_FILE}"
88-99-# curl a URL and fail if the request fails.
1010-function curl_cmd_get {
1111- curl --fail --silent --show-error "$@"
1212-}
1313-1414-# curl a URL and fail if the request fails.
1515-function curl_cmd_post {
1616- curl --fail --silent --show-error --request POST --header "Content-Type: application/json" "$@"
1717-}
1818-1919-# curl a URL but do not fail if the request fails.
2020-function curl_cmd_post_nofail {
2121- curl --silent --show-error --request POST --header "Content-Type: application/json" "$@"
2222-}
2323-2424-# The subcommand to run.
2525-SUBCOMMAND="${1:-}"
2626-2727-#
2828-# account list
2929-#
3030-if [[ "${SUBCOMMAND}" == "list" ]]; then
3131- DIDS="$(curl_cmd_get \
3232- "https://${PDS_HOSTNAME}/xrpc/com.atproto.sync.listRepos?limit=100" | jq --raw-output '.repos[].did'
3333- )"
3434- OUTPUT='[{"handle":"Handle","email":"Email","did":"DID"}'
3535- for did in ${DIDS}; do
3636- ITEM="$(curl_cmd_get \
3737- --user "admin:${PDS_ADMIN_PASSWORD}" \
3838- "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.getAccountInfo?did=${did}"
3939- )"
4040- OUTPUT="${OUTPUT},${ITEM}"
4141- done
4242- OUTPUT="${OUTPUT}]"
4343- echo "${OUTPUT}" | jq --raw-output '.[] | [.handle, .email, .did] | @tsv' | column --table
4444-4545-#
4646-# account create
4747-#
4848-elif [[ "${SUBCOMMAND}" == "create" ]]; then
4949- EMAIL="${2:-}"
5050- HANDLE="${3:-}"
5151-5252- if [[ "${EMAIL}" == "" ]]; then
5353- read -p "Enter an email address (e.g. alice@${PDS_HOSTNAME}): " EMAIL
5454- fi
5555- if [[ "${HANDLE}" == "" ]]; then
5656- read -p "Enter a handle (e.g. alice.${PDS_HOSTNAME}): " HANDLE
5757- fi
5858-5959- if [[ "${EMAIL}" == "" || "${HANDLE}" == "" ]]; then
6060- echo "ERROR: missing EMAIL and/or HANDLE parameters." >/dev/stderr
6161- echo "Usage: $0 ${SUBCOMMAND} <EMAIL> <HANDLE>" >/dev/stderr
6262- exit 1
6363- fi
6464-6565- PASSWORD="$(openssl rand -base64 30 | tr -d "=+/" | cut -c1-24)"
6666- INVITE_CODE="$(curl_cmd_post \
6767- --user "admin:${PDS_ADMIN_PASSWORD}" \
6868- --data '{"useCount": 1}' \
6969- "https://${PDS_HOSTNAME}/xrpc/com.atproto.server.createInviteCode" | jq --raw-output '.code'
7070- )"
7171- RESULT="$(curl_cmd_post_nofail \
7272- --data "{\"email\":\"${EMAIL}\", \"handle\":\"${HANDLE}\", \"password\":\"${PASSWORD}\", \"inviteCode\":\"${INVITE_CODE}\"}" \
7373- "https://${PDS_HOSTNAME}/xrpc/com.atproto.server.createAccount"
7474- )"
7575-7676- DID="$(echo $RESULT | jq --raw-output '.did')"
7777- if [[ "${DID}" != did:* ]]; then
7878- ERR="$(echo ${RESULT} | jq --raw-output '.message')"
7979- echo "ERROR: ${ERR}" >/dev/stderr
8080- echo "Usage: $0 ${SUBCOMMAND} <EMAIL> <HANDLE>" >/dev/stderr
8181- exit 1
8282- fi
8383-8484- echo
8585- echo "Account created successfully!"
8686- echo "-----------------------------"
8787- echo "Handle : ${HANDLE}"
8888- echo "DID : ${DID}"
8989- echo "Password : ${PASSWORD}"
9090- echo "-----------------------------"
9191- echo "Save this password, it will not be displayed again."
9292- echo
9393-9494-#
9595-# account delete
9696-#
9797-elif [[ "${SUBCOMMAND}" == "delete" ]]; then
9898- DID="${2:-}"
9999-100100- if [[ "${DID}" == "" ]]; then
101101- echo "ERROR: missing DID parameter." >/dev/stderr
102102- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
103103- exit 1
104104- fi
105105-106106- if [[ "${DID}" != did:* ]]; then
107107- echo "ERROR: DID parameter must start with \"did:\"." >/dev/stderr
108108- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
109109- exit 1
110110- fi
111111-112112- echo "This action is permanent."
113113- read -r -p "Are you sure you'd like to delete ${DID}? [y/N] " response
114114- if [[ ! "${response}" =~ ^([yY][eE][sS]|[yY])$ ]]; then
115115- exit 0
116116- fi
117117-118118- curl_cmd_post \
119119- --user "admin:${PDS_ADMIN_PASSWORD}" \
120120- --data "{\"did\": \"${DID}\"}" \
121121- "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.deleteAccount" >/dev/null
122122-123123- echo "${DID} deleted"
124124-125125-#
126126-# account takedown
127127-#
128128-elif [[ "${SUBCOMMAND}" == "takedown" ]]; then
129129- DID="${2:-}"
130130- TAKEDOWN_REF="$(date +%s)"
131131-132132- if [[ "${DID}" == "" ]]; then
133133- echo "ERROR: missing DID parameter." >/dev/stderr
134134- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
135135- exit 1
136136- fi
137137-138138- if [[ "${DID}" != did:* ]]; then
139139- echo "ERROR: DID parameter must start with \"did:\"." >/dev/stderr
140140- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
141141- exit 1
142142- fi
143143-144144- PAYLOAD="$(cat <<EOF
145145- {
146146- "subject": {
147147- "\$type": "com.atproto.admin.defs#repoRef",
148148- "did": "${DID}"
149149- },
150150- "takedown": {
151151- "applied": true,
152152- "ref": "${TAKEDOWN_REF}"
153153- }
154154- }
155155-EOF
156156-)"
157157-158158- curl_cmd_post \
159159- --user "admin:${PDS_ADMIN_PASSWORD}" \
160160- --data "${PAYLOAD}" \
161161- "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.updateSubjectStatus" >/dev/null
162162-163163- echo "${DID} taken down"
164164-165165-#
166166-# account untakedown
167167-#
168168-elif [[ "${SUBCOMMAND}" == "untakedown" ]]; then
169169- DID="${2:-}"
170170-171171- if [[ "${DID}" == "" ]]; then
172172- echo "ERROR: missing DID parameter." >/dev/stderr
173173- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
174174- exit 1
175175- fi
176176-177177- if [[ "${DID}" != did:* ]]; then
178178- echo "ERROR: DID parameter must start with \"did:\"." >/dev/stderr
179179- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
180180- exit 1
181181- fi
182182-183183- PAYLOAD=$(cat <<EOF
184184- {
185185- "subject": {
186186- "\$type": "com.atproto.admin.defs#repoRef",
187187- "did": "${DID}"
188188- },
189189- "takedown": {
190190- "applied": false
191191- }
192192- }
193193-EOF
194194-)
195195-196196- curl_cmd_post \
197197- --user "admin:${PDS_ADMIN_PASSWORD}" \
198198- --data "${PAYLOAD}" \
199199- "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.updateSubjectStatus" >/dev/null
200200-201201- echo "${DID} untaken down"
202202-#
203203-# account reset-password
204204-#
205205-elif [[ "${SUBCOMMAND}" == "reset-password" ]]; then
206206- DID="${2:-}"
207207- PASSWORD="$(openssl rand -base64 30 | tr -d "=+/" | cut -c1-24)"
208208-209209- if [[ "${DID}" == "" ]]; then
210210- echo "ERROR: missing DID parameter." >/dev/stderr
211211- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
212212- exit 1
213213- fi
214214-215215- if [[ "${DID}" != did:* ]]; then
216216- echo "ERROR: DID parameter must start with \"did:\"." >/dev/stderr
217217- echo "Usage: $0 ${SUBCOMMAND} <DID>" >/dev/stderr
218218- exit 1
219219- fi
220220-221221- curl_cmd_post \
222222- --user "admin:${PDS_ADMIN_PASSWORD}" \
223223- --data "{ \"did\": \"${DID}\", \"password\": \"${PASSWORD}\" }" \
224224- "https://${PDS_HOSTNAME}/xrpc/com.atproto.admin.updateAccountPassword" >/dev/null
225225-226226- echo
227227- echo "Password reset for ${DID}"
228228- echo "New password: ${PASSWORD}"
229229- echo
230230-231231-else
232232- echo "Unknown subcommand: ${SUBCOMMAND}" >/dev/stderr
233233- exit 1
234234-fi
···11-#!/bin/bash
22-set -o errexit
33-set -o nounset
44-set -o pipefail
55-66-# This script is used to display help information for the pdsadmin command.
77-cat <<HELP
88-pdsadmin help
99---
1010-update
1111- Update to the latest PDS version.
1212- e.g. pdsadmin update
1313-1414-account
1515- list
1616- List accounts
1717- e.g. pdsadmin account list
1818- create <EMAIL> <HANDLE>
1919- Create a new account
2020- e.g. pdsadmin account create alice@example.com alice.example.com
2121- delete <DID>
2222- Delete an account specified by DID.
2323- e.g. pdsadmin account takedown did:plc:xyz123abc456
2424- takedown <DID>
2525- Takedown an account specified by DID.
2626- e.g. pdsadmin account takedown did:plc:xyz123abc456
2727- untakedown <DID>
2828- Remove a takedown an account specified by DID.
2929- e.g. pdsadmin account takedown did:plc:xyz123abc456
3030- password-reset <DID>
3131- Reset a password for an account specified by DID.
3232- e.g. pdsadmin account reset-password did:plc:xyz123abc456
3333-3434-request-crawl [<RELAY HOST>]
3535- Request a crawl from a relay host.
3636- e.g. pdsadmin request-crawl bsky.network
3737-3838-create-invite-code
3939- Create a new invite code.
4040- e.g. pdsadmin create-invite-code
4141-4242-help
4343- Display this help information.
4444-4545-HELP
···11-#!/bin/bash
22-set -o errexit
33-set -o nounset
44-set -o pipefail
55-66-PDS_DATADIR="/pds"
77-COMPOSE_FILE="${PDS_DATADIR}/compose.yaml"
88-COMPOSE_URL="https://raw.githubusercontent.com/bluesky-social/pds/main/compose.yaml"
99-1010-# TODO: allow the user to specify a version to update to.
1111-TARGET_VERSION="${1:-}"
1212-1313-COMPOSE_TEMP_FILE="${COMPOSE_FILE}.tmp"
1414-1515-echo "* Downloading PDS compose file"
1616-curl \
1717- --silent \
1818- --show-error \
1919- --fail \
2020- --output "${COMPOSE_TEMP_FILE}" \
2121- "${COMPOSE_URL}"
2222-2323-if cmp --quiet "${COMPOSE_FILE}" "${COMPOSE_TEMP_FILE}"; then
2424- echo "PDS is already up to date"
2525- rm --force "${COMPOSE_TEMP_FILE}"
2626- exit 0
2727-fi
2828-2929-echo "* Updating PDS"
3030-mv "${COMPOSE_TEMP_FILE}" "${COMPOSE_FILE}"
3131-3232-echo "* Restarting PDS"
3333-systemctl restart pds
3434-3535-cat <<MESSAGE
3636-PDS has been updated
3737----------------------
3838-Check systemd logs: journalctl --unit pds
3939-Check container logs: docker logs pds
4040-4141-MESSAGE
-30
pdsadmin.sh
···11-#!/bin/bash
22-set -o errexit
33-set -o nounset
44-set -o pipefail
55-66-PDSADMIN_BASE_URL="https://raw.githubusercontent.com/bluesky-social/pds/main/pdsadmin"
77-88-# Command to run.
99-COMMAND="${1:-help}"
1010-shift || true
1111-1212-# Ensure the user is root, since it's required for most commands.
1313-if [[ "${EUID}" -ne 0 ]]; then
1414- echo "ERROR: This script must be run as root"
1515- exit 1
1616-fi
1717-1818-# Download the script, if it exists.
1919-SCRIPT_URL="${PDSADMIN_BASE_URL}/${COMMAND}.sh"
2020-SCRIPT_FILE="$(mktemp /tmp/pdsadmin.${COMMAND}.XXXXXX)"
2121-2222-if ! curl --fail --silent --show-error --location --output "${SCRIPT_FILE}" "${SCRIPT_URL}"; then
2323- echo "ERROR: ${COMMAND} not found"
2424- exit 2
2525-fi
2626-2727-chmod +x "${SCRIPT_FILE}"
2828-if "${SCRIPT_FILE}" "$@"; then
2929- rm --force "${SCRIPT_FILE}"
3030-fi