tokono.ma
A music player that connects to your cloud/distributed storage.
1# Per-source listing progress
2
3## Context
4
5When tracks are processed, there are two phases: **listing** (discovering tracks from each source) and **metadata** (fetching tags/stats per track). Currently only the metadata phase reports progress. The listing phase can be slow (e.g. enumerating an S3 bucket) but appears as a black box — the UI has no visibility into it.
6
7The goal is to show per-source progress during listing, so the user sees something like "Listing sources (2/3)..." before the metadata phase kicks in.
8
9## Architecture challenge
10
11The call chain crosses three worker boundaries:
12
13```
14process-tracks worker →(RPC)→ input configurator worker →(RPC)→ per-scheme workers
15```
16
17- `process-tracks/worker.js` calls `input.list(cachedTracks)` as one opaque RPC call
18- `configurator/input/worker.js` receives it, fans out to per-scheme workers via `Promise.all`
19- Each scheme worker (s3, https, opensubsonic) does the actual listing
20
21The `announce`/`listen` mechanism only communicates between a worker and **its owning element**. So announcements from the input configurator worker go to the `dc-input` element, not to the process-tracks element or worker. This means progress must be surfaced through the element layer.
22
23## Plan
24
25### 1. Add listing progress signal to input configurator worker
26
27**File:** `src/components/configurator/input/worker.js`
28
29- Import `signal`, `effect`, `announce`
30- Add module-level signal: `const $listingProgress = signal({ processed: 0, total: 0 })`
31- In the `list()` function, count groups and update `$listingProgress` as each source completes:
32
33```js
34export async function list({ data, ports }) {
35 const groups = await groupConsult({ data, ports });
36 const entries = Object.values(groups);
37
38 $listingProgress.value = { processed: 0, total: entries.length };
39 let processed = 0;
40
41 const promises = entries.map(async ({ available, scheme, tracks }) => {
42 if (!available) { ... }
43 const result = await input.list(tracks);
44 processed++;
45 $listingProgress.value = { processed, total: entries.length };
46 return result;
47 });
48
49 const nested = await Promise.all(promises);
50 return nested.flat(1);
51}
52```
53
54- In `ostiary()`, announce the signal and expose it via RPC:
55
56```js
57ostiary((context) => {
58 rpc(context, { ..., listingProgress: $listingProgress.get });
59 effect(() => announce("listingProgress", $listingProgress.value, context));
60});
61```
62
63### 2. Expose listing progress from input configurator element
64
65**File:** `src/components/configurator/input/element.js`
66
67- Import `signal` from `@common/signal.js` and `listen` from `@common/worker.js`
68- Add a `#listingProgress` signal and a public `listingProgress` getter
69- Add `connectedCallback()` to set up `listen("listingProgress", ...)` on `this.workerLink()`
70
71### 3. Proxy listing progress through the input orchestrator
72
73**File:** `src/components/orchestrator/input/element.js`
74
75- Add a `listingProgress` getter that delegates to `this.input.listingProgress()`
76
77### 4. Add a phase to process-tracks progress
78
79**File:** `src/components/orchestrator/process-tracks/types.d.ts`
80
81- Extend the `Progress` type with a `phase` field:
82
83```ts
84export type Progress = {
85 phase: "listing" | "metadata";
86 processed: number;
87 total: number;
88};
89```
90
91**File:** `src/components/orchestrator/process-tracks/element.js`
92
93- The element already queries `input-selector` (the `do-input` element)
94- Add an `effect()` that watches `this.input.listingProgress()` while `isProcessing` is true, and maps it into `#progress` with `phase: "listing"`
95- When the worker's own metadata progress arrives (via the existing `listen("progress", ...)`), set it with `phase: "metadata"`
96
97**File:** `src/components/orchestrator/process-tracks/worker.js`
98
99- Update `$progress` initial value to include `phase: "metadata"` (the worker only knows about metadata)
100
101### 5. Update the UI to show the phase
102
103**File:** `src/themes/webamp/configurators/input/element.js`
104
105- In `#renderProcessingProgress()`, read `phase` from the progress object
106- Show "Listing sources (2/3)..." during listing phase
107- Show "Gathering metadata (5/10)..." during metadata phase (current behavior)
108
109## Files changed (6)
110
111| File | Change | Difficulty |
112|------|--------|-----------|
113| `src/components/configurator/input/worker.js` | Add `$listingProgress` signal, update `list()`, announce in `ostiary` | Low-medium |
114| `src/components/configurator/input/element.js` | Add signal, `connectedCallback`, `listen`, expose getter | Low |
115| `src/components/orchestrator/input/element.js` | Proxy `listingProgress` getter | Trivial |
116| `src/components/orchestrator/process-tracks/element.js` | Watch input's listing progress, merge into phased progress | Medium |
117| `src/components/orchestrator/process-tracks/worker.js` | Add `phase` to progress signal | Trivial |
118| `src/components/orchestrator/process-tracks/types.d.ts` | Add `phase` to `Progress` type | Trivial |
119| `src/themes/webamp/configurators/input/element.js` | Render phase-aware progress text | Low |
120
121**Overall difficulty: Medium.** Follows existing patterns (`announce`/`listen`, signals, proxied getters) throughout. The trickiest part is step 4 — having the process-tracks element watch the input element's listing progress and merge it with the worker's metadata progress into a single unified signal.
122
123## Verification
124
125- Add an S3 source with enough files to make listing take a visible amount of time
126- Open the Overview tab in the input configurator
127- Observe "Listing sources (X/Y)..." during listing, transitioning to "Gathering metadata (X/Y)..." during metadata extraction
128- Confirm the progress bar advances during both phases