A CLI for publishing standard.site documents to ATProto sequoia.pub
standard site lexicon cli publishing

Slug and Frontmatter Updates #3

closed opened by stevedylan.dev targeting main from chore/frontmatter-config-updates

Overview#

This PR fixes issues raised in #3

UX Changes#

  • Subdirectory support: Posts in nested folders like a/b/post.md can now preserve their full path in URLs (set slugSource: "path")
  • Frontmatter slug support: Use a field from your frontmatter (like slug or url) to define the path (set slugSource: "frontmatter")
  • Hugo index.md support: Remove /index suffix from paths automatically (set removeIndexFromSlug: true)
  • Custom text content: Use a frontmatter field like summary for the AT Protocol textContent instead of the full markdown body (set textContentField: "summary")
  • Description now works: The description field from frontmatter is now actually published to AT Protocol records (was a bug , never added to records before)

Example config using frontmatter slugs:

  {
    "siteUrl": "https://example.com",
    "contentDir": "content/posts",
    "pathPrefix": "/blog",
    "publicationUri": "at://did:plc:.../site.standard.publication/...",
    "slugSource": "frontmatter",
    "slugField": "url"
  }

Example frontmatter:

---
title: "My Post Title"
date: 2024-01-15
url: "2024/my-custom-slug"
summary: "A brief description of this post"
---

This would create a document with path /blog/2024/my-custom-slug

File Updates#

  1. src/lib/types.ts
  • Added new config options to PublisherConfig:
  • slugSource: "filename" | "path" | "frontmatter" (default: "filename")
  • slugField: string for frontmatter field when using slugSource: "frontmatter"
  • removeIndexFromSlug: boolean to strip /index or /_index suffixes
  • textContentField: frontmatter field to use for textContent instead of markdown body
  • Added rawFrontmatter: Record<string, unknown> to BlogPost
  1. src/lib/markdown.ts
  • Updated parseFrontmatter() to return rawFrontmatter
  • Added getSlugFromOptions() function supporting three modes (filename, path, frontmatter)
  • Added ScanOptions interface for cleaner option passing
  • Updated scanContentDirectory() to accept new slug options and pass rawFrontmatter
  1. src/lib/atproto.ts
  • Added description field to DocumentRecord interface
  • Fixed createDocument() to add description to record when available
  • Fixed updateDocument() to add description to record when available
  • Added textContentField support to both functions
  1. src/commands/publish.ts
  • Updated to pass new config options to scanContentDirectory()
  1. src/commands/sync.ts
  • Updated to pass new config options to scanContentDirectory()
  • Fixed path matching to use configured pathPrefix
  1. src/lib/config.ts
  • Updated generateConfigTemplate() to include new options
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:ia2zdnhjaokf5lazhxrmj6eu/sh.tangled.repo.pull/3mdpsypgmvl22
+80 -53
Interdiff #0 โ†’ #1
+1
packages/cli/src/commands/publish.ts
··· 221 221 contentHash, 222 222 atUri, 223 223 lastPublished: new Date().toISOString(), 224 + slug: post.slug, 224 225 }; 225 226 } catch (error) { 226 227 const errorMessage = error instanceof Error ? error.message : String(error);
packages/cli/src/commands/sync.ts

This file has not been changed.

packages/cli/src/lib/atproto.ts

This file has not been changed.

packages/cli/src/lib/config.ts

This file has not been changed.

+51 -3
packages/cli/src/lib/markdown.ts
··· 34 34 const raw: Record<string, unknown> = {}; 35 35 const lines = frontmatterStr.split("\n"); 36 36 37 - for (const line of lines) { 37 + let i = 0; 38 + while (i < lines.length) { 39 + const line = lines[i]; 40 + if (line === undefined) { 41 + i++; 42 + continue; 43 + } 38 44 const sepIndex = line.indexOf(separator); 39 - if (sepIndex === -1) continue; 45 + if (sepIndex === -1) { 46 + i++; 47 + continue; 48 + } 40 49 41 50 const key = line.slice(0, sepIndex).trim(); 42 51 let value = line.slice(sepIndex + 1).trim(); ··· 49 58 value = value.slice(1, -1); 50 59 } 51 60 52 - // Handle arrays (simple case for tags) 61 + // Handle inline arrays (simple case for tags) 53 62 if (value.startsWith("[") && value.endsWith("]")) { 54 63 const arrayContent = value.slice(1, -1); 55 64 raw[key] = arrayContent 56 65 .split(",") 57 66 .map((item) => item.trim().replace(/^["']|["']$/g, "")); 67 + } else if (value === "" && !isToml) { 68 + // Check for YAML-style multiline array (key with no value followed by - items) 69 + const arrayItems: string[] = []; 70 + let j = i + 1; 71 + while (j < lines.length) { 72 + const nextLine = lines[j]; 73 + if (nextLine === undefined) { 74 + j++; 75 + continue; 76 + } 77 + // Check if line is a list item (starts with whitespace and -) 78 + const listMatch = nextLine.match(/^\s+-\s*(.*)$/); 79 + if (listMatch && listMatch[1] !== undefined) { 80 + let itemValue = listMatch[1].trim(); 81 + // Remove quotes if present 82 + if ( 83 + (itemValue.startsWith('"') && itemValue.endsWith('"')) || 84 + (itemValue.startsWith("'") && itemValue.endsWith("'")) 85 + ) { 86 + itemValue = itemValue.slice(1, -1); 87 + } 88 + arrayItems.push(itemValue); 89 + j++; 90 + } else if (nextLine.trim() === "") { 91 + // Skip empty lines within the array 92 + j++; 93 + } else { 94 + // Hit a new key or non-list content 95 + break; 96 + } 97 + } 98 + if (arrayItems.length > 0) { 99 + raw[key] = arrayItems; 100 + i = j; 101 + continue; 102 + } else { 103 + raw[key] = value; 104 + } 58 105 } else if (value === "true") { 59 106 raw[key] = true; 60 107 } else if (value === "false") { ··· 62 109 } else { 63 110 raw[key] = value; 64 111 } 112 + i++; 65 113 } 66 114 67 115 // Apply field mappings to normalize to standard PostFrontmatter fields
+1
packages/cli/src/lib/types.ts
··· 67 67 contentHash: string; 68 68 atUri?: string; 69 69 lastPublished?: string; 70 + slug?: string; // The generated slug for this post (used by inject command) 70 71 } 71 72 72 73 export interface PublicationRecord {
+27 -50
packages/cli/src/commands/inject.ts
··· 44 44 // Load state to get atUri mappings 45 45 const state = await loadState(configDir); 46 46 47 - // Generic filenames where the slug is the parent directory, not the filename 48 - // Covers: SvelteKit (+page), Astro/Hugo (index), Next.js (page), etc. 49 - const genericFilenames = new Set([ 50 - "+page", 51 - "index", 52 - "_index", 53 - "page", 54 - "readme", 55 - ]); 56 - 57 - // Build a map of slug/path to atUri from state 58 - const pathToAtUri = new Map<string, string>(); 47 + // Build a map of slug to atUri from state 48 + // The slug is stored in state by the publish command, using the configured slug options 49 + const slugToAtUri = new Map<string, string>(); 59 50 for (const [filePath, postState] of Object.entries(state.posts)) { 60 - if (postState.atUri) { 61 - // Extract slug from file path (e.g., ./content/blog/my-post.md -> my-post) 62 - let basename = path.basename(filePath, path.extname(filePath)); 51 + if (postState.atUri && postState.slug) { 52 + // Use the slug stored in state (computed by publish with config options) 53 + slugToAtUri.set(postState.slug, postState.atUri); 63 54 64 - // If the filename is a generic convention name, use the parent directory as slug 65 - if (genericFilenames.has(basename.toLowerCase())) { 66 - // Split path and filter out route groups like (blog-article) 67 - const pathParts = filePath 68 - .split(/[/\\]/) 69 - .filter((p) => p && !(p.startsWith("(") && p.endsWith(")"))); 70 - // The slug should be the second-to-last part (last is the filename) 71 - if (pathParts.length >= 2) { 72 - const slug = pathParts[pathParts.length - 2]; 73 - if (slug && slug !== "." && slug !== "content" && slug !== "routes" && slug !== "src") { 74 - basename = slug; 75 - } 76 - } 55 + // Also add the last segment for simpler matching 56 + // e.g., "40th-puzzle-box/what-a-gift" -> also map "what-a-gift" 57 + const lastSegment = postState.slug.split("/").pop(); 58 + if (lastSegment && lastSegment !== postState.slug) { 59 + slugToAtUri.set(lastSegment, postState.atUri); 77 60 } 78 - 79 - pathToAtUri.set(basename, postState.atUri); 80 - 81 - // Also add variations that might match HTML file paths 82 - // e.g., /blog/my-post, /posts/my-post, my-post/index 83 - const dirName = path.basename(path.dirname(filePath)); 84 - // Skip route groups and common directory names 85 - if (dirName !== "." && dirName !== "content" && !(dirName.startsWith("(") && dirName.endsWith(")"))) { 86 - pathToAtUri.set(`${dirName}/${basename}`, postState.atUri); 87 - } 61 + } else if (postState.atUri) { 62 + // Fallback for older state files without slug field 63 + // Extract slug from file path (e.g., ./content/blog/my-post.md -> my-post) 64 + const basename = path.basename(filePath, path.extname(filePath)); 65 + slugToAtUri.set(basename.toLowerCase(), postState.atUri); 88 66 } 89 67 } 90 68 91 - if (pathToAtUri.size === 0) { 69 + if (slugToAtUri.size === 0) { 92 70 log.warn( 93 71 "No published posts found in state. Run 'sequoia publish' first.", 94 72 ); 95 73 return; 96 74 } 97 75 98 - log.info(`Found ${pathToAtUri.size} published posts in state`); 76 + log.info(`Found ${slugToAtUri.size} slug mappings from published posts`); 99 77 100 78 // Scan for HTML files 101 79 const htmlFiles = await glob("**/*.html", { ··· 125 103 let atUri: string | undefined; 126 104 127 105 // Strategy 1: Direct basename match (e.g., my-post.html -> my-post) 128 - atUri = pathToAtUri.get(htmlBasename); 106 + atUri = slugToAtUri.get(htmlBasename); 129 107 130 - // Strategy 2: Directory name for index.html (e.g., my-post/index.html -> my-post) 108 + // Strategy 2: For index.html, try the directory path 109 + // e.g., posts/40th-puzzle-box/what-a-gift/index.html -> 40th-puzzle-box/what-a-gift 131 110 if (!atUri && htmlBasename === "index" && htmlDir !== ".") { 132 - const slug = path.basename(htmlDir); 133 - atUri = pathToAtUri.get(slug); 111 + // Try full directory path (for nested subdirectories) 112 + atUri = slugToAtUri.get(htmlDir); 134 113 135 - // Also try parent/slug pattern 114 + // Also try just the last directory segment 136 115 if (!atUri) { 137 - const parentDir = path.dirname(htmlDir); 138 - if (parentDir !== ".") { 139 - atUri = pathToAtUri.get(`${path.basename(parentDir)}/${slug}`); 140 - } 116 + const lastDir = path.basename(htmlDir); 117 + atUri = slugToAtUri.get(lastDir); 141 118 } 142 119 } 143 120 144 121 // Strategy 3: Full path match (e.g., blog/my-post.html -> blog/my-post) 145 122 if (!atUri && htmlDir !== ".") { 146 - atUri = pathToAtUri.get(`${htmlDir}/${htmlBasename}`); 123 + atUri = slugToAtUri.get(`${htmlDir}/${htmlBasename}`); 147 124 } 148 125 149 126 if (!atUri) {

History

5 rounds 1 comment
sign up or login to add to the discussion
7 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
chore: added linting and formatting
chore: linting updates
chore: refactored to use fallback approach if frontmatter.slugField is provided or not
expand 1 comment

For some reason tangled doesn't like rebasing, so going to close this out and open a fresh PR

closed without merging
6 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
chore: added linting and formatting
chore: linting updates
expand 0 comments
4 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
expand 0 comments
3 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
expand 0 comments
1 commit
expand
chore: resolved action items from issue #3
expand 0 comments