elasticsearch-based configurable generic appview for prototyping ideas

esav live explanation

rimar1337 a45f1f84 1460bea0

Changed files
+122 -6
+122 -6
readme.md
··· 1 1 # ESAV: ElasticSearch AppView 2 - terrible i know right lmao 2 + Jetstream plugged into ElasticSearch with a queryable api 3 + (also now supports live queries) 3 4 4 - send ES queries to `/xrpc/com.example.prototypeESQuery` either POST or GET q= 5 + ## Queries 6 + ### the boring one 7 + send ES queries to `/xrpc/party.whey.esav.esQuery` either POST or GET q= 5 8 6 - change config in `config.json` 9 + the format is just a normal elasticsearch query, any of them should work 7 10 8 - run `deno task dev` to start 11 + ### the cooler Live one (sync over websocket) 12 + ESAV Live is a real-time indexing and query service ESAV. 9 13 10 - you need elasticsearch configured and running first though, good luck with that 14 + the websocket provides a live stream of new document at:// URIs that match a specific query 11 15 12 - ## ForumTest, built with ESAV 16 + send Live queries to `wss://{your domain}/xrpc/party.whey.esav.esSync` 17 + 18 + once a connection is open, the client must send a subscription request (the live query itelf). this message is a JSON object containing the query that defines the feed you are interested in. the query format is basically a dumbed down Elasticsearch Query DSL (stripped to the few stuff thats actually implemented, and thatll produce a deterministic order / result) 19 + 20 + for example, to subscribe to all statuses from the `xyz.statusphere.status` collection, send the JSON request bleow: 21 + 22 + ```json 23 + { 24 + "type": "subscribe", 25 + "queryId": "statusphere-main-page", 26 + "esquery": { 27 + "query": { 28 + "term": { 29 + "$metadata.collection": "xyz.statusphere.status" 30 + } 31 + }, 32 + "sort": [ 33 + { 34 + "$metadata.indexedAt": "desc" 35 + } 36 + ], 37 + "size": 100 38 + } 39 + } 40 + ``` 41 + 42 + 43 + and then server will send back JSON messages as new documents matching your query are indexed. the primary message type youll get is a `query-delta` which will contain an array of new document URIs for each active query, and also the set of new or modified documents to store. 44 + 45 + example server message: 46 + 47 + ```json 48 + { 49 + "type": "query-delta", 50 + "documents": { 51 + "at://did:plc:mn45tewwnse5btfftvd3powc/xyz.statusphere.status/3lvsr2zqf3k2n": { 52 + "cid": "bafyreidym6ad6k7grbz7nrqrnddht2rwxlnimbgbfddthrmftv7mbfk7gy", 53 + "doc": { 54 + "$metadata.uri": "at://did:plc:mn45tewwnse5btfftvd3powc/xyz.statusphere.status/3lvsr2zqf3k2n", 55 + "$metadata.cid": "bafyreidym6ad6k7grbz7nrqrnddht2rwxlnimbgbfddthrmftv7mbfk7gy", 56 + "$metadata.indexedAt": "2025-08-07T12:40:09.426Z", 57 + "$metadata.did": "did:plc:mn45tewwnse5btfftvd3powc", 58 + "$metadata.collection": "xyz.statusphere.status", 59 + "$metadata.rkey": "3lvsr2zqf3k2n", 60 + "status": "👍", 61 + "$raw": { 62 + "$type": "xyz.statusphere.status", 63 + "createdAt": "2025-08-07T12:40:08.602Z", 64 + "status": "👍" 65 + } 66 + } 67 + } 68 + }, 69 + "queries": { 70 + "statusphere-main-page": { 71 + "ecid": "bafyreidsxtbtnwce2o72wrwtwhcvc7olosdtvb2i4g6ezbs4kaxn3v3qna", 72 + "result": [ 73 + "at://did:plc:mn45tewwnse5btfftvd3powc/xyz.statusphere.status/3lvsr2zqf3k2n", 74 + "at://did:plc:mn45tewwnse5btfftvd3powc/xyz.statusphere.status/3lvsov4tatf2n", 75 + "at://did:plc:mn45tewwnse5btfftvd3powc/xyz.statusphere.status/3lvsouyvrbr2t" 76 + ] 77 + } 78 + } 79 + } 80 + ``` 81 + 82 + in here, only one document is given because this is not the initial response after a registered live query. or alternatively, a live query was registered using a still-valid "ecid" value (the window is like 5 minutes so its really only for dropped connections). 83 + 84 + your application should listen for these messages and prepend the new URIs to its list of statuses, creating the real-time effect 85 + 86 + ### helper functions 87 + 88 + because we need to sometimes resolve their did from handle, or handle from did 89 + 90 + and also because despite your usage of a custom lexicon, we still need to fetch app.bsky.actor.profile for their pfp right 91 + 92 + fetch `https://esav.whey.party/xrpc/party.whey.esav.resolveIdentity` 93 + 94 + with these params: 95 + - `did`: The DID of the user. 96 + - `handle`: The handle of the user. 97 + - `includePfp` (optional): `true` to include the PFP URL. 98 + 99 + **Example Request:** 100 + 101 + `https://esav.whey.party/xrpc/party.whey.esav.resolveIdentity?did=did:web:did12.whey.party&includePfp=true` 102 + 103 + gets me 104 + 105 + **Example Response (`200 OK`):** 106 + 107 + ```json 108 + { 109 + "did":"did:web:did12.whey.party", 110 + "pdsUrl":"https://pds-nd.whey.party", 111 + "handle":"dw.whey.party", 112 + "pfp":"https://pds-nd.whey.party/xrpc/com.atproto.sync.getBlob?did=did:web:did12.whey.party&cid=bafkreibesqir3254ee3natyi3tadhmcjqyyeukoiqxsctsru7z3j4ws4mm" 113 + } 114 + ``` 115 + 116 + > **Note on Performance**: This endpoint is called frequently. It is highly recommended to implement a client-side cache (e.g., a simple JavaScript `Map` or object) to store profile data keyed by DID. This will prevent your application from making redundant network requests for the same user profile, significantly improving performance. 117 + 118 + 119 + ## stuff built with ESAV 120 + 121 + ### Statusphere ESAV Live 122 + an example usage of ESAV Live with the "hello world!" of atproto 123 + 124 + check it out here -> [https://statusphere.whey.party/](https://statusphere.whey.party/) 125 + repo: [https://tangled.sh/@whey.party/statusphere-esav-live](https://tangled.sh/@whey.party/statusphere-esav-live) 126 + 127 + ### ForumTest 128 + atproto forum thing test 13 129 14 130 check it out here -> [https://forumtest.whey.party](https://forumtest.whey.party) 15 131 repo: [https://tangled.sh/@whey.party/forumtest](https://tangled.sh/@whey.party/forumtest)