PDS Admin tool make it easier to moderate your PDS with labels

Fork 4

Add an `ozone-tag` action to surface external labels in an Ozone queue #18

Merged

spotchi.eurosky.social 3mo ago

targeting main from spotchi.eurosky.social/label-watcher:ozone-tag

Merged opened by and assigned to

spotchi.eurosky.social 3mo ago targeting main from spotchi.eurosky.social/label-watcher:ozone-tag

For PDS operators who also run their own Ozone instance: this adds a third label action, ozone-tag, that emits a tools.ozone.moderation.defs#modEventTag event for the labeled account into a configured Ozone.

Why: Ozone's queryStatuses can only filter on tags/excludeTags, not labels, and a subject only shows up in the queue at all if it has a status row in that Ozone's own database — which labels from external labelers never create. Converting a watched label into a tag bridges that: the account lands in the moderation queue and can be filtered with ?tags=<tag>.

Config #

[ozone]
serviceDid = "did:plc:yourozoneservice"   # #atproto_labeler fragment optional
pdsHost = "your-pds.com"                  # PDS the mod account lives on
handle = "mod.your-pds.com"
appPassword = "xxxx-xxxx-xxxx-xxxx"

[labeler.bluesky.labels.takedown]
label_name = "!takedown"
action = "ozone-tag"
tag = "ext:bluesky:takedown"    # default: ext:<labeler-key>:<label-value>
# requireWatchedRepo = false    # set to tag accounts beyond your own PDSs

Commits 6

refactor: move label actions behind per-action handlers built by a factory

+278 -200

refactor: turn labels_applied into an event log keyed on the label event

+282 -49

perf: throttle labeler cursor writes to every 100 messages

+34 -12

feat: add ozone-tag action to surface labels in an Ozone queue

+496 -49

fix: return the handleNewLabel promise to the label queue

+3 -3

fix: match labels on label_name instead of the config key

+5 -2

Comments

v0 3mo ago

pds.dad 3mo ago

Nice! Thank you for the PR. Looks like mostly the idea is to let moderation actions from other labelers bubble up to ozone?

Def fine and on board with that. Only thing I was a bit concerned about is the drop of a foreign key on did of labels_applied

Does it matter to bubble them up if the user is not on the PDS?

Does it have to record that a label was applied locally if it's just bubbling up?

May be another label like ozone_events_sent if a durable storage for "this has been sent or not" may be better?

src/handlers/handleNewLabel.ts:79

spotchi.eurosky.social 3mo ago

Yep, that's the idea — labels from labelers you configure are forwarded to Ozone so that you can review them independently and apply your own action.

The drop of the Foreign key is relaxing the constraint on the labels applied relating to accounts on the PDS. I am actually not using that option at the moment (mostly just watching takedowns on our PDS), but could see a case where we need our own decision beyond just the PDS. Also I didn't see it used except for existence checks for inserts/updates.

What is the role of the labels_applied table? Is it akin to an audit log? If so perhaps it does make sense to have a different table for ozone actions sent so that those are not reapplied, and that is something that is ozone specific.

pds.dad 2mo ago

Yes, mostly just an audit and to help with duplicate labels on replay of the labeler's web socket.

Mostly just concerned about any look ups since it runs on any watched dids or I guess in ozone mode any new labels. Looking at the code again I was really kind of focusing on labels that linked to a did on the repo and not even worrying about the target which could be an at://uri as well.

So may just be using the target/seq/labler maybe to make a unique key (like a xxhash of it) and that's the lookup for that table instead of the did and should fix both? Still keep the did on the table if it's needed but can break the foreign key

spotchi.eurosky.social 2mo ago

Labels Applied table lookup and re-processing #

I've changed the use of the labels applied as an event log, it is append-only. The unique key is (labeler, seq, uri, label), I decided not to use a hash column as I don't think it would bring any benefit and xxhash would be one more dependency.

A negation or re-application is a new row with its own seq rather than an in-place update, so the audit history survives.

export const labelsApplied = sqliteTable(
  "labels_applied",
  {
    id: integer("id").primaryKey({ autoIncrement: true }),
    seq: integer("seq"),
    labeler: text("labeler").notNull(),
    uri: text("uri"),
    did: text("did").notNull(),
    label: text("label").notNull(),
    action: text("action").notNull(),
    negated: integer("negated", { mode: "boolean" }).default(false).notNull(),
    dateApplied: integer("date_applied", { mode: "timestamp" }).notNull(),
  },
  (t) => [
    uniqueIndex("labels_applied_event_idx").on(t.labeler, t.seq, t.uri, t.label),
    index("labels_applied_did_idx").on(t.did),
  ],
);

Your concern about the lookup should be addressed with the per-did index, all queries should hit it.

I am using it to understand whether a particular label was already applied. We record the label in the labels applied table at the end of the processing of actions. This means the actions themselves need to be "idempotent" in the sense that if an action is applied and an error occurs afterwards, or the process crashes and doesn't finish the batch processing, replaying that label must succeed (or re-doing the action must be acceptable).

Takedown : checks to take_down_issued_date
Ozone tagging : Ozone's externalId (now the same string as the event key) means the tag emission is idempotent as it's deduped on the Ozone side
Emails actually resend Also fixes a latent collision in the old key: two record labels on different records of the same repo used to overwrite each other's row, and uri actually gets populated now.

Migration is additive (ADD COLUMN seq + two indexes, no table rebuild), auto-applies on startup like the others. Existing rows keep seq/uri NULL — NULLs are distinct in SQLite unique indexes so they can't collide with new rows or each other (tested the migration over a db with legacy data). Old code also runs fine against the new schema, so rolling back doesn't need a schema rollback.

handleNewLabel refactor #

There is also a small refactor of handleNewLabel (735ce88) that moves the actions behind per-action handlers. Each Action handler lives in its own file, and are created once at startup. This makes the code for the label handler more readable.

Config #

Versions

Labels Applied table lookup and re-processing #

handleNewLabel refactor #