update READMEs and dev docs

HACKING.md

···
 
 ## git repo contents
 
-Commands (run with, eg, `go run ./cmd/bigsky`):
+Run with, eg, `go run ./cmd/bigsky`):
 
 - `cmd/bigsky`: BGS+indexer daemon
 - `cmd/palomar`: search indexer and query servcie (OpenSearch)
 - `cmd/gosky`: client CLI for talking to a PDS
 - `cmd/lexgen`: codegen tool for lexicons (Lexicon JSON to Go package)
-- `cmd/laputa`: PDS daemon
+- `cmd/laputa`: partial PDS daemon (not usable or under development)
 - `cmd/stress`: connects to local/default PDS and creates a ton of random posts
 - `cmd/beemo`: slack bot for moderation reporting (Bluesky Moderation Observer)
 - `cmd/fakermaker`: helper to generate fake accounts and content for testing
+- `cmd/supercollider`: event stream load generation tool
+- `cmd/sonar`: event stream monitoring tool
 - `gen`: dev tool to run CBOR type codegen
 
 Packages:
 
 - `api`: mostly output of lexgen (codegen) for lexicons: structs, CBOR marshaling. some higher-level code, and a PLC client (may rename)
-    - `api/atprot`: generated types for `com.atproto` lexicon
+    - `api/atproto`: generated types for `com.atproto` lexicon
     - `api/bsky`: generated types for `app.bsky` lexicon
+- `atproto/crypto`: crytographic helpers (signing, key generation and serialization)
+- `atproto/syntax`: string types and parsers for identifiers, datetimes, etc
+- `atproto/identity`: DID and handle resolution
 - `bgs`: server implementation for crawling, etc
 - `carstore`: library for storing repo data in CAR files on disk, plus a metadata SQL db
 - `events`: types, codegen CBOR helpers, and persistence for event feeds
···
 - `util`: a few common definitions (may rename)
 - `xrpc`: XRPC client (not server) helpers
 
-Other:
-
-- `testscripts`: shell scripts that run gosky (CLI client) against local PDS
 
-
-## jargon
+## Jargon
 
 - BGS: Big Graph Service (or Server), which centrals crawls/consumes content from "all" PDSs and re-broadcasts as a firehose
 - PDS: Personal Data Server (or Service), which stores user atproto repositories and acts as a user agent in the network
 - CLI: Command Line Tool
 - CBOR: a binary serialization format, smilar to JSON
-- PLC: "placeholder" DID provider
-- DID: Decentralized IDentifier, a flexible W3C specification for persistent identifiers in URI form (eg, "did:plc:abcd1234")
+- PLC: "placeholder" DID provider, see <https://web.plc.directory>
+- DID: Decentralized IDentifier, a flexible W3C specification for persistent identifiers in URI form (eg, `did:plc:abcd1234`)
 - XRPC: atproto convention for HTTP GET and POST endpoints specified by namespaced Lexicon schemas
 - CAR: simple file format for storing binary content-addressed blocks/blobs, sort of like .tar files
 - CID: content identifier for binary blobs, basically a flexible encoding of hash values
 - MST: Merkle Search Tree, a key/value map data structure using content addressed nodes
 
 
-## lexicon and CBOR marshaling code generation
+## Lexicon and CBOR code generation
 
 `gen/main.go` has a list of types internal to packages in this repo which need CBOR helper codegen. If you edit those types, or update the listed types/packages, re-run codegen like:
 
···
     # then generate
     go run ./gen
 
-To run codegen for new or updated Lexicons, using lexgen, first place (or git
-checkout) the JSON lexicon files at `../atproto/`.
-Then, in *this* repository (indigo), run commands like:
+To run codegen for new or updated Lexicons, using lexgen, first place (or git checkout) the JSON lexicon files at `../atproto/`. Then, in *this* repository (indigo), run commands like:
 
     go run ./cmd/lexgen/ --package bsky --prefix app.bsky --outdir api/bsky ../atproto/lexicons/app/bsky/
     go run ./cmd/lexgen/ --package atproto --prefix com.atproto --outdir api/atproto ../atproto/lexicons/com/atproto/
 
 You may want to delete all the codegen files before re-generating, to detect deleted files.
 
-It can require some manual munging between the lexgen step and a later `go run ./gen` to make sure things compile at least temporarily; otherwise the `gen` will not run.
-In some cases, you might also need to add new types to ./gen/main.go.
+It can require some manual munging between the lexgen step and a later `go run ./gen` to make sure things compile at least temporarily; otherwise the `gen` will not run. In some cases, you might also need to add new types to `./gen/main.go`.
 
-To generate server stubs and handlers, push them in a temporary directory
-first, then merge changes in to the actual PDS code:
+To generate server stubs and handlers, push them in a temporary directory first, then merge changes in to the actual PDS code:
 
     mkdir tmppds
     go run ./cmd/lexgen/ --package pds --gen-server --types-import com.atproto:github.com/bluesky-social/indigo/api/atproto --types-import app.bsky:github.com/bluesky-social/indigo/api/bsky --outdir tmppds --gen-handlers ../atproto/lexicons
 
 
-## tips and tricks
+## Tips and Tricks
 
 When debugging websocket streams, the `websocat` tool (rust) can be helpful. CBOR binary is sort of mangled in to text by default. Eg:
 
···
     GOLOG_LOG_LEVEL=info go run ./cmd/pds
 
 
-## gosky basic usage
+## `gosky` basic usage
 
 Running against local typescript PDS in `dev-env` mode:
 
 	# as "alice" user
 	go run ./cmd/gosky/ --pds http://localhost:2583 createSession alice.test hunter2 > bsky.auth
 
-The `bsky.auth` file is the default place that `gosky` and other client
-commands will look for auth info.
+The `bsky.auth` file is the default place that `gosky` and other client commands will look for auth info.
 
 
-## slack report bot basic usage
+## Integrated Development
 
-You need an admin token, slack webhook URL, and auth file (see gosky above).
-The auth file isn't actually used, only the admin token.
+Sometimes it is helpful to run a PLC, PDS, BGS, labelmaker, and other components, all locally on your laptop, across languages. This section describes one setup for this.
 
-    # configure a slack webhook
-    export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T028K87/B04NBDB/oWbsHasdf23r2d
-
-    # example pulling admin token out of `pass` password manager
-    export ATP_AUTH_ADMIN_PASSWORD=`pass bsky/pds-admin-staging | head -n1`
-
-    # example just setting admin token directly
-    export ATP_AUTH_ADMIN_PASSWORD="someinsecurething123"
-
-    # run the bot
-    GOLOG_LOG_LEVEL=debug go run ./cmd/beemo/ --pds https://pds.staging.example.com --auth bsky.auth notify-reports
-
-## integrated development
-
-Sometimes it is helpful to run a PLC, PDS, BGS, labelmaker, and other
-components, all locally on your laptop, across languages. This section
-describes one setup for this.
-
-First, you need PostgreSQL running locally. This could be via docker, or the
-following commands assume some kind of debian/ubuntu setup with a postgres
-server package installed and running.
+First, you need PostgreSQL running locally. This could be via docker, or the following commands assume some kind of debian/ubuntu setup with a postgres server package installed and running.
 
 Create a user and databases for PLC+PDS:
 

README.md

···
 
 ![photo](https://static.bnewbold.net/tmp/indigo_serac.jpeg)
 
-indigo: golang code for Bluesky's atproto services
-==================================================
+indigo: atproto libraries and services in golang
+================================================
 
 Some Bluesky software is developed in Typescript, and lives in the [bluesky-social/atproto](https://github.com/bluesky-social/atproto) repository. Some is developed in Go, and lives here.
 
-<p align="center"><img src="https://static.bnewbold.net/tmp/under_construction_bar.gif" /></p>
+Services implemented in this repository:
 
-Everything in this repository is an work in progress. Features and "Lexicons" may be removed or updated, software interfaces broken, etc.
+* **`bigsky`** ([README](./cmd/bigsky/README.md)): "Big Graph Service" (BGS) reference implementation, running at `bsky.network`
+* **`palomar`** ([README](./cmd/palomar/README.md)): fulltext search service for <https://bsky.app>
 
-We are developing in the open, but not ready to accept or review significant contributions. Keep checking back!
+
+## Development Quickstart
+
+All the packages in this repository are under active development. Features and software interfaces have not stabilized and may break or be removed.
 
 <p align="center"><img src="https://static.bnewbold.net/tmp/under_construction_bar.gif" /></p>
 
+First, you will need the Go toolchain installed. We develop using the latest stable version of the language.
+
+The Makefile provides wrapper commands for basic development:
+
+    make build
+    make test
+    make fmt
+    make lint
+
+Individual commands can be run like:
+
+    go run ./cmd/bigsky
+
+The [HACKING](./HACKING.md) file has a list of commands and packages in this repository and some other development tips.
+
 
 ## What is atproto?
 
···
 
 The Authenticated Transfer Protocol ("ATP" or "atproto") is a decentralized social media protocol, developed by [Bluesky PBC](https://blueskyweb.xyz). Learn more at:
 
-- [Protocol Documentation](https://atproto.com/docs)
-- [Overview Guide](https://atproto.com/guides/overview) 👈 Good place to start
+- [Overview and Guides](https://atproto.com/guides/overview) 👈 Best starting point
+- [Github Discussions](https://github.com/bluesky-social/atproto/discussions) 👈 Great place to ask questions
+- [Protocol Specifications](https://atproto.com/specs/atp)
 - [Blogpost on self-authenticating data structures](https://blueskyweb.xyz/blog/3-6-2022-a-self-authenticating-social-protocol)
 
-
-## Development
+The Bluesky Social application encompasses a set of schemas and APIs built in the overall AT Protocol framework. The namespace for these "Lexicons" is `app.bsky.*`.
 
-First, you will need the Go toolchain installed. We develop using the latest stable version of the language.
 
-The Makefile provides wrapper commands for basic development:
+## Contributions
 
-    make build
-    make test
-    make fmt
-    make lint
+We are working in the open, but not ready to actively collaborate on larger contributions to this codebase.
 
-Individual commands can be run like:
+Please at least open an issue ahead of time, *before* starting any non-trivial work that you hope to get reviewed or merged to this repo.
 
-    go run ./cmd/bigsky
 
 ## Are you a developer interested in building on atproto?
 

cmd/beemo/README.md

···
+
+## beemo: Slack notification bot for moderation reports
+
+You need an admin token, slack webhook URL, and auth file (see gosky docs).
+The auth file isn't actually used, only the admin token.
+
+    # configure a slack webhook
+    export SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T028K87/B04NBDB/oWbsHasdf23r2d
+
+    # example pulling admin token out of `pass` password manager
+    export ATP_AUTH_ADMIN_PASSWORD=`pass bsky/pds-admin-staging | head -n1`
+
+    # example just setting admin token directly
+    export ATP_AUTH_ADMIN_PASSWORD="someinsecurething123"
+
+    # run the bot
+    GOLOG_LOG_LEVEL=debug go run ./cmd/beemo/ --pds https://pds.staging.example.com --auth bsky.auth notify-reports