tokono.ma
A music player that connects to your cloud/distributed storage.
1# HTTPS directory listing (`https+json`)
2
3## Context
4
5Some HTTPS servers expose a directory listing API: appending `?ls` to a directory URL returns a JSON array describing its contents:
6
7```json
8[
9 { "name": "my-directory", "type": "directory", "mtime": "2022-10-07T00:53:50Z" },
10 { "name": "my_file.tar.gz", "type": "file", "mtime": "2022-09-27T22:44:34Z", "size": 332 }
11]
12```
13
14Rather than bolting this onto the existing HTTPS input (which is designed for individual file URLs), this feature is implemented as a separate `https+json` input with its own URI scheme. Users add a server URL (e.g. `https+json://music.example.com`) and the input automatically discovers all audio files under it — recursively traversing subdirectories.
15
16The existing HTTPS input is left untouched.
17
18## Plan
19
20### 1. Create `src/components/input/https-json/constants.js`
21
22```js
23export const SCHEME = "https+json";
24```
25
26### 2. Create `src/components/input/https-json/common.js`
27
28Follows the same structure as `https/common.js`. Key additions are `buildURI` and `listDirectory`.
29
30**`parseURI(uriString)`** — validates the `https+json:` scheme and returns components:
31
32```js
33export function parseURI(uriString) {
34 try {
35 const url = new URL(uriString);
36 if (url.protocol !== "https+json:") return undefined;
37
38 return {
39 // The real HTTPS URL (for fetch calls)
40 url: "https:" + uriString.slice("https+json:".length),
41 domain: url.hostname,
42 host: url.host,
43 path: url.pathname,
44 };
45 } catch {
46 return undefined;
47 }
48}
49```
50
51**`buildURI(host, path)`** — constructs an `https+json://` URI:
52
53```js
54export function buildURI(host, path = "/") {
55 return `https+json://${host}${path}`;
56}
57```
58
59**`listDirectory(httpsUrl)`** — fetches `url?ls`, parses the JSON listing, and recursively collects audio file URLs. Returns `https+json://` URIs (not plain `https://`):
60
61```js
62export async function listDirectory(httpsUrl) {
63 let entries;
64
65 try {
66 const response = await fetch(httpsUrl + "?ls");
67 if (!response.ok) return [];
68 entries = await response.json();
69 } catch {
70 return [];
71 }
72
73 if (!Array.isArray(entries)) return [];
74
75 const results = await Promise.all(
76 entries.map(async (entry) => {
77 if (entry.type === "file") {
78 if (!isAudioFile(entry.name)) return [];
79 const fileHttpsUrl = httpsUrl + entry.name;
80 // Return as https+json:// URI
81 return [fileHttpsUrl.replace(/^https:/, "https+json:")];
82 } else {
83 return listDirectory(httpsUrl + entry.name + "/");
84 }
85 }),
86 );
87
88 return results.flat(1);
89}
90```
91
92Include `groupTracksByHost`, `groupUrisByHost`, `hostsFromTracks` — same implementations as in `https/common.js` but using `parseURI` from this module. Include `consultHostCached` — same implementation, caches per host.
93
94### 3. Create `src/components/input/https-json/worker.js`
95
96Five standard actions. Mirrors the S3 worker pattern for `list()`.
97
98**`consult`** — same shape as HTTPS: scheme-only returns `undetermined`, full URI checks host reachability via `consultHostCached`.
99
100**`detach`** — same shape as HTTPS: groups by host, removes the target host's group.
101
102**`groupConsult`** — same shape as HTTPS: groups URIs by host, checks each host once.
103
104**`list(cachedTracks)`** — the key new logic:
105
106```js
107export async function list(cachedTracks = []) {
108 // Build a URI → track cache for preserving metadata across relists.
109 /** @type {Record<string, Track>} */
110 const cacheByUri = {};
111 cachedTracks.forEach((t) => { cacheByUri[t.uri] = t; });
112
113 // Group by host; derive the scan root per host.
114 // If placeholder tracks exist for a host, use their path as the root.
115 // Otherwise, find the common path prefix of the audio file tracks.
116 const groups = groupTracksByHost(cachedTracks);
117
118 const promises = Object.values(groups).map(async ({ host, tracks }) => {
119 const root = scanRoot(tracks); // "/" or a shared directory prefix
120 const rootHttpsUrl = `https://${host}${root}`;
121 const now = new Date().toISOString();
122
123 const uris = await listDirectory(rootHttpsUrl);
124
125 let discovered = uris.map((uri) => {
126 const cached = cacheByUri[uri];
127 /** @type {Track} */
128 return {
129 $type: "sh.diffuse.output.track",
130 id: cached?.id ?? TID.now(),
131 createdAt: cached?.createdAt ?? now,
132 updatedAt: cached?.updatedAt ?? now,
133 stats: cached?.stats,
134 tags: cached?.tags,
135 uri,
136 };
137 });
138
139 if (!discovered.length) {
140 discovered = [{
141 $type: "sh.diffuse.output.track",
142 id: TID.now(),
143 createdAt: now,
144 updatedAt: now,
145 kind: "placeholder",
146 uri: buildURI(host, root),
147 }];
148 }
149
150 return discovered;
151 });
152
153 return (await Promise.all(promises)).flat(1);
154}
155```
156
157`scanRoot(tracks)` is a small helper (defined in `worker.js` or `common.js`) that:
158- Returns the path of the first placeholder-kind track if one exists
159- Otherwise returns the longest common path prefix of all track paths, trimmed to the last `/`
160
161**`resolve({ uri })`** — strips the scheme prefix and returns the `https://` URL with a one-year expiry (same as HTTPS input):
162
163```js
164export async function resolve({ uri }) {
165 const parsed = parseURI(uri);
166 if (!parsed) return undefined;
167
168 const expiresAt = Math.round(Date.now() / 1000) + 60 * 60 * 24 * 365;
169 return { url: parsed.url, expiresAt };
170}
171```
172
173### 4. Create `src/components/input/https-json/element.js`
174
175Minimal — same shape as `https/element.js`. `sources()` returns unique hosts from tracks.
176
177```js
178class HttpsJsonInput extends DiffuseElement {
179 static NAME = "diffuse/input/https-json";
180 static WORKER_URL = "components/input/https-json/worker.js";
181
182 SCHEME = SCHEME;
183
184 constructor() {
185 super();
186 this.proxy = this.workerProxy();
187 this.consult = this.proxy.consult;
188 this.detach = this.proxy.detach;
189 this.groupConsult = this.proxy.groupConsult;
190 this.list = this.proxy.list;
191 this.resolve = this.proxy.resolve;
192 }
193
194 sources(tracks) {
195 return Object.values(hostsFromTracks(tracks)).map((host) => ({
196 label: host,
197 uri: buildURI(host),
198 }));
199 }
200}
201
202export const CLASS = HttpsJsonInput;
203export const NAME = "di-https-json";
204
205customElements.define(NAME, CLASS);
206```
207
208### 5. Register in `src/components/orchestrator/input/element.js`
209
210Add the import and the element to the rendered template:
211
212```js
213import "~/components/input/https-json/element.js";
214// ...
215render({ html }) {
216 return html`
217 <dc-input>
218 <di-https></di-https>
219 <di-https-json></di-https-json>
220 <di-opensubsonic></di-opensubsonic>
221 <di-s3></di-s3>
222 </dc-input>
223 `;
224}
225```
226
227## Files changed (5)
228
229| File | Change | Difficulty |
230|------|--------|------------|
231| `src/components/input/https-json/constants.js` | New — define `SCHEME = "https+json"` | Trivial |
232| `src/components/input/https-json/common.js` | New — `parseURI`, `buildURI`, `listDirectory`, grouping helpers, `consultHostCached` | Low |
233| `src/components/input/https-json/worker.js` | New — all 5 actions; `list()` does recursive `?ls` fetching | Low-medium |
234| `src/components/input/https-json/element.js` | New — `HttpsJsonInput` element, `di-https-json` | Trivial |
235| `src/components/orchestrator/input/element.js` | Add import + `<di-https-json>` to template | Trivial |
236
237**Overall difficulty: Low-medium.** All patterns are direct copies or adaptations of the existing HTTPS and S3 inputs. The only genuinely new logic is `listDirectory()` (recursive `?ls` fetch) and `scanRoot()` (derive listing root from cached tracks).
238
239## Verification
240
241- Add `https+json://music.example.com` as a source and trigger a library refresh
242- Confirm audio files from the server's directory tree appear as tracks
243- Confirm cached metadata (tags, stats) is preserved across relists
244- Add a server with no audio files — confirm a placeholder track is produced
245- Confirm `resolve()` returns a working `https://` URL for each track
246- Confirm `di-https` (plain HTTPS) is unaffected