A CLI for publishing standard.site documents to ATProto sequoia.pub
standard site lexicon cli publishing

Slug and Frontmatter Updates #3

closed opened by stevedylan.dev targeting main from chore/frontmatter-config-updates

Overview#

This PR fixes issues raised in #3

UX Changes#

  • Subdirectory support: Posts in nested folders like a/b/post.md can now preserve their full path in URLs (set slugSource: "path")
  • Frontmatter slug support: Use a field from your frontmatter (like slug or url) to define the path (set slugSource: "frontmatter")
  • Hugo index.md support: Remove /index suffix from paths automatically (set removeIndexFromSlug: true)
  • Custom text content: Use a frontmatter field like summary for the AT Protocol textContent instead of the full markdown body (set textContentField: "summary")
  • Description now works: The description field from frontmatter is now actually published to AT Protocol records (was a bug , never added to records before)

Example config using frontmatter slugs:

  {
    "siteUrl": "https://example.com",
    "contentDir": "content/posts",
    "pathPrefix": "/blog",
    "publicationUri": "at://did:plc:.../site.standard.publication/...",
    "slugSource": "frontmatter",
    "slugField": "url"
  }

Example frontmatter:

---
title: "My Post Title"
date: 2024-01-15
url: "2024/my-custom-slug"
summary: "A brief description of this post"
---

This would create a document with path /blog/2024/my-custom-slug

File Updates#

  1. src/lib/types.ts
  • Added new config options to PublisherConfig:
  • slugSource: "filename" | "path" | "frontmatter" (default: "filename")
  • slugField: string for frontmatter field when using slugSource: "frontmatter"
  • removeIndexFromSlug: boolean to strip /index or /_index suffixes
  • textContentField: frontmatter field to use for textContent instead of markdown body
  • Added rawFrontmatter: Record<string, unknown> to BlogPost
  1. src/lib/markdown.ts
  • Updated parseFrontmatter() to return rawFrontmatter
  • Added getSlugFromOptions() function supporting three modes (filename, path, frontmatter)
  • Added ScanOptions interface for cleaner option passing
  • Updated scanContentDirectory() to accept new slug options and pass rawFrontmatter
  1. src/lib/atproto.ts
  • Added description field to DocumentRecord interface
  • Fixed createDocument() to add description to record when available
  • Fixed updateDocument() to add description to record when available
  • Added textContentField support to both functions
  1. src/commands/publish.ts
  • Updated to pass new config options to scanContentDirectory()
  1. src/commands/sync.ts
  • Updated to pass new config options to scanContentDirectory()
  • Fixed path matching to use configured pathPrefix
  1. src/lib/config.ts
  • Updated generateConfigTemplate() to include new options
Labels

None yet.

assignee

None yet.

Participants 1
AT URI
at://did:plc:ia2zdnhjaokf5lazhxrmj6eu/sh.tangled.repo.pull/3mdpsypgmvl22
+156 -12
Diff #0
+7 -1
packages/cli/src/commands/publish.ts
··· 87 // Scan for posts 88 const s = spinner(); 89 s.start("Scanning for posts..."); 90 - const posts = await scanContentDirectory(contentDir, config.frontmatter, config.ignore); 91 s.stop(`Found ${posts.length} posts`); 92 93 // Determine which posts need publishing
··· 87 // Scan for posts 88 const s = spinner(); 89 s.start("Scanning for posts..."); 90 + const posts = await scanContentDirectory(contentDir, { 91 + frontmatterMapping: config.frontmatter, 92 + ignorePatterns: config.ignore, 93 + slugSource: config.slugSource, 94 + slugField: config.slugField, 95 + removeIndexFromSlug: config.removeIndexFromSlug, 96 + }); 97 s.stop(`Found ${posts.length} posts`); 98 99 // Determine which posts need publishing
+10 -3
packages/cli/src/commands/sync.ts
··· 90 91 // Scan local posts 92 s.start("Scanning local content..."); 93 - const localPosts = await scanContentDirectory(contentDir, config.frontmatter); 94 s.stop(`Found ${localPosts.length} local posts`); 95 96 // Build a map of path -> local post for matching 97 - // Document path is like /posts/my-post-slug 98 const postsByPath = new Map<string, typeof localPosts[0]>(); 99 for (const post of localPosts) { 100 - const postPath = `/posts/${post.slug}`; 101 postsByPath.set(postPath, post); 102 } 103
··· 90 91 // Scan local posts 92 s.start("Scanning local content..."); 93 + const localPosts = await scanContentDirectory(contentDir, { 94 + frontmatterMapping: config.frontmatter, 95 + ignorePatterns: config.ignore, 96 + slugSource: config.slugSource, 97 + slugField: config.slugField, 98 + removeIndexFromSlug: config.removeIndexFromSlug, 99 + }); 100 s.stop(`Found ${localPosts.length} local posts`); 101 102 // Build a map of path -> local post for matching 103 + // Document path is like /posts/my-post-slug (or custom pathPrefix) 104 + const pathPrefix = config.pathPrefix || "/posts"; 105 const postsByPath = new Map<string, typeof localPosts[0]>(); 106 for (const post of localPosts) { 107 + const postPath = `${pathPrefix}/${post.slug}`; 108 postsByPath.set(postPath, post); 109 } 110
+25 -2
packages/cli/src/lib/atproto.ts
··· 171 ): Promise<string> { 172 const pathPrefix = config.pathPrefix || "/posts"; 173 const postPath = `${pathPrefix}/${post.slug}`; 174 - const textContent = stripMarkdownForText(post.content); 175 const publishDate = new Date(post.frontmatter.publishDate); 176 177 const record: Record<string, unknown> = { 178 $type: "site.standard.document", 179 title: post.frontmatter.title, ··· 184 canonicalUrl: `${config.siteUrl}${postPath}`, 185 }; 186 187 if (coverImage) { 188 record.coverImage = coverImage; 189 } ··· 219 220 const pathPrefix = config.pathPrefix || "/posts"; 221 const postPath = `${pathPrefix}/${post.slug}`; 222 - const textContent = stripMarkdownForText(post.content); 223 const publishDate = new Date(post.frontmatter.publishDate); 224 225 const record: Record<string, unknown> = { 226 $type: "site.standard.document", 227 title: post.frontmatter.title, ··· 232 canonicalUrl: `${config.siteUrl}${postPath}`, 233 }; 234 235 if (coverImage) { 236 record.coverImage = coverImage; 237 } ··· 266 textContent: string; 267 publishedAt: string; 268 canonicalUrl?: string; 269 coverImage?: BlobObject; 270 tags?: string[]; 271 location?: string;
··· 171 ): Promise<string> { 172 const pathPrefix = config.pathPrefix || "/posts"; 173 const postPath = `${pathPrefix}/${post.slug}`; 174 const publishDate = new Date(post.frontmatter.publishDate); 175 176 + // Determine textContent: use configured field from frontmatter, or fallback to markdown body 177 + let textContent: string; 178 + if (config.textContentField && post.rawFrontmatter?.[config.textContentField]) { 179 + textContent = String(post.rawFrontmatter[config.textContentField]); 180 + } else { 181 + textContent = stripMarkdownForText(post.content); 182 + } 183 + 184 const record: Record<string, unknown> = { 185 $type: "site.standard.document", 186 title: post.frontmatter.title, ··· 191 canonicalUrl: `${config.siteUrl}${postPath}`, 192 }; 193 194 + if (post.frontmatter.description) { 195 + record.description = post.frontmatter.description; 196 + } 197 + 198 if (coverImage) { 199 record.coverImage = coverImage; 200 } ··· 230 231 const pathPrefix = config.pathPrefix || "/posts"; 232 const postPath = `${pathPrefix}/${post.slug}`; 233 const publishDate = new Date(post.frontmatter.publishDate); 234 235 + // Determine textContent: use configured field from frontmatter, or fallback to markdown body 236 + let textContent: string; 237 + if (config.textContentField && post.rawFrontmatter?.[config.textContentField]) { 238 + textContent = String(post.rawFrontmatter[config.textContentField]); 239 + } else { 240 + textContent = stripMarkdownForText(post.content); 241 + } 242 + 243 const record: Record<string, unknown> = { 244 $type: "site.standard.document", 245 title: post.frontmatter.title, ··· 250 canonicalUrl: `${config.siteUrl}${postPath}`, 251 }; 252 253 + if (post.frontmatter.description) { 254 + record.description = post.frontmatter.description; 255 + } 256 + 257 if (coverImage) { 258 record.coverImage = coverImage; 259 } ··· 288 textContent: string; 289 publishedAt: string; 290 canonicalUrl?: string; 291 + description?: string; 292 coverImage?: BlobObject; 293 tags?: string[]; 294 location?: string;
+20
packages/cli/src/lib/config.ts
··· 76 pdsUrl?: string; 77 frontmatter?: FrontmatterMapping; 78 ignore?: string[]; 79 }): string { 80 const config: Record<string, unknown> = { 81 siteUrl: options.siteUrl, ··· 112 config.ignore = options.ignore; 113 } 114 115 return JSON.stringify(config, null, 2); 116 } 117
··· 76 pdsUrl?: string; 77 frontmatter?: FrontmatterMapping; 78 ignore?: string[]; 79 + slugSource?: "filename" | "path" | "frontmatter"; 80 + slugField?: string; 81 + removeIndexFromSlug?: boolean; 82 + textContentField?: string; 83 }): string { 84 const config: Record<string, unknown> = { 85 siteUrl: options.siteUrl, ··· 116 config.ignore = options.ignore; 117 } 118 119 + if (options.slugSource && options.slugSource !== "filename") { 120 + config.slugSource = options.slugSource; 121 + } 122 + 123 + if (options.slugField && options.slugField !== "slug") { 124 + config.slugField = options.slugField; 125 + } 126 + 127 + if (options.removeIndexFromSlug) { 128 + config.removeIndexFromSlug = options.removeIndexFromSlug; 129 + } 130 + 131 + if (options.textContentField) { 132 + config.textContentField = options.textContentField; 133 + } 134 + 135 return JSON.stringify(config, null, 2); 136 } 137
+89 -6
packages/cli/src/lib/markdown.ts
··· 7 export function parseFrontmatter(content: string, mapping?: FrontmatterMapping): { 8 frontmatter: PostFrontmatter; 9 body: string; 10 } { 11 // Support multiple frontmatter delimiters: 12 // --- (YAML) - Jekyll, Astro, most SSGs ··· 102 // Always preserve atUri (internal field) 103 frontmatter.atUri = raw.atUri; 104 105 - return { frontmatter: frontmatter as unknown as PostFrontmatter, body }; 106 } 107 108 export function getSlugFromFilename(filename: string): string { ··· 112 .replace(/\s+/g, "-"); 113 } 114 115 export async function getContentHash(content: string): Promise<string> { 116 const encoder = new TextEncoder(); 117 const data = encoder.encode(content); ··· 129 return false; 130 } 131 132 export async function scanContentDirectory( 133 contentDir: string, 134 - frontmatterMapping?: FrontmatterMapping, 135 ignorePatterns: string[] = [] 136 ): Promise<BlogPost[]> { 137 const patterns = ["**/*.md", "**/*.mdx"]; 138 const posts: BlogPost[] = []; 139 ··· 145 146 for (const relativePath of files) { 147 // Skip files matching ignore patterns 148 - if (shouldIgnore(relativePath, ignorePatterns)) { 149 continue; 150 } 151 ··· 153 const rawContent = await fs.readFile(filePath, "utf-8"); 154 155 try { 156 - const { frontmatter, body } = parseFrontmatter(rawContent, frontmatterMapping); 157 - const filename = path.basename(relativePath); 158 - const slug = getSlugFromFilename(filename); 159 160 posts.push({ 161 filePath, ··· 163 frontmatter, 164 content: body, 165 rawContent, 166 }); 167 } catch (error) { 168 console.error(`Error parsing ${relativePath}:`, error);
··· 7 export function parseFrontmatter(content: string, mapping?: FrontmatterMapping): { 8 frontmatter: PostFrontmatter; 9 body: string; 10 + rawFrontmatter: Record<string, unknown>; 11 } { 12 // Support multiple frontmatter delimiters: 13 // --- (YAML) - Jekyll, Astro, most SSGs ··· 103 // Always preserve atUri (internal field) 104 frontmatter.atUri = raw.atUri; 105 106 + return { frontmatter: frontmatter as unknown as PostFrontmatter, body, rawFrontmatter: raw }; 107 } 108 109 export function getSlugFromFilename(filename: string): string { ··· 113 .replace(/\s+/g, "-"); 114 } 115 116 + export interface SlugOptions { 117 + slugSource?: "filename" | "path" | "frontmatter"; 118 + slugField?: string; 119 + removeIndexFromSlug?: boolean; 120 + } 121 + 122 + export function getSlugFromOptions( 123 + relativePath: string, 124 + rawFrontmatter: Record<string, unknown>, 125 + options: SlugOptions = {} 126 + ): string { 127 + const { slugSource = "filename", slugField = "slug", removeIndexFromSlug = false } = options; 128 + 129 + let slug: string; 130 + 131 + switch (slugSource) { 132 + case "path": 133 + // Use full relative path without extension 134 + slug = relativePath 135 + .replace(/\.mdx?$/, "") 136 + .toLowerCase() 137 + .replace(/\s+/g, "-"); 138 + break; 139 + 140 + case "frontmatter": 141 + // Use frontmatter field (slug or url) 142 + const frontmatterValue = rawFrontmatter[slugField] || rawFrontmatter.slug || rawFrontmatter.url; 143 + if (frontmatterValue && typeof frontmatterValue === "string") { 144 + // Remove leading slash if present 145 + slug = frontmatterValue.replace(/^\//, "").toLowerCase().replace(/\s+/g, "-"); 146 + } else { 147 + // Fallback to filename if frontmatter field not found 148 + slug = getSlugFromFilename(path.basename(relativePath)); 149 + } 150 + break; 151 + 152 + case "filename": 153 + default: 154 + slug = getSlugFromFilename(path.basename(relativePath)); 155 + break; 156 + } 157 + 158 + // Remove /index or /_index suffix if configured 159 + if (removeIndexFromSlug) { 160 + slug = slug.replace(/\/_?index$/, ""); 161 + } 162 + 163 + return slug; 164 + } 165 + 166 export async function getContentHash(content: string): Promise<string> { 167 const encoder = new TextEncoder(); 168 const data = encoder.encode(content); ··· 180 return false; 181 } 182 183 + export interface ScanOptions { 184 + frontmatterMapping?: FrontmatterMapping; 185 + ignorePatterns?: string[]; 186 + slugSource?: "filename" | "path" | "frontmatter"; 187 + slugField?: string; 188 + removeIndexFromSlug?: boolean; 189 + } 190 + 191 export async function scanContentDirectory( 192 contentDir: string, 193 + frontmatterMappingOrOptions?: FrontmatterMapping | ScanOptions, 194 ignorePatterns: string[] = [] 195 ): Promise<BlogPost[]> { 196 + // Handle both old signature (frontmatterMapping, ignorePatterns) and new signature (options) 197 + let options: ScanOptions; 198 + if (frontmatterMappingOrOptions && ('slugSource' in frontmatterMappingOrOptions || 'frontmatterMapping' in frontmatterMappingOrOptions || 'ignorePatterns' in frontmatterMappingOrOptions)) { 199 + options = frontmatterMappingOrOptions as ScanOptions; 200 + } else { 201 + // Old signature: (contentDir, frontmatterMapping?, ignorePatterns?) 202 + options = { 203 + frontmatterMapping: frontmatterMappingOrOptions as FrontmatterMapping | undefined, 204 + ignorePatterns, 205 + }; 206 + } 207 + 208 + const { 209 + frontmatterMapping, 210 + ignorePatterns: ignore = [], 211 + slugSource, 212 + slugField, 213 + removeIndexFromSlug, 214 + } = options; 215 + 216 const patterns = ["**/*.md", "**/*.mdx"]; 217 const posts: BlogPost[] = []; 218 ··· 224 225 for (const relativePath of files) { 226 // Skip files matching ignore patterns 227 + if (shouldIgnore(relativePath, ignore)) { 228 continue; 229 } 230 ··· 232 const rawContent = await fs.readFile(filePath, "utf-8"); 233 234 try { 235 + const { frontmatter, body, rawFrontmatter } = parseFrontmatter(rawContent, frontmatterMapping); 236 + const slug = getSlugFromOptions(relativePath, rawFrontmatter, { 237 + slugSource, 238 + slugField, 239 + removeIndexFromSlug, 240 + }); 241 242 posts.push({ 243 filePath, ··· 245 frontmatter, 246 content: body, 247 rawContent, 248 + rawFrontmatter, 249 }); 250 } catch (error) { 251 console.error(`Error parsing ${relativePath}:`, error);
+5
packages/cli/src/lib/types.ts
··· 18 identity?: string; // Which stored identity to use (matches identifier) 19 frontmatter?: FrontmatterMapping; // Custom frontmatter field mappings 20 ignore?: string[]; // Glob patterns for files to ignore (e.g., ["_index.md", "**/drafts/**"]) 21 } 22 23 export interface Credentials { ··· 41 frontmatter: PostFrontmatter; 42 content: string; 43 rawContent: string; 44 } 45 46 export interface BlobRef {
··· 18 identity?: string; // Which stored identity to use (matches identifier) 19 frontmatter?: FrontmatterMapping; // Custom frontmatter field mappings 20 ignore?: string[]; // Glob patterns for files to ignore (e.g., ["_index.md", "**/drafts/**"]) 21 + slugSource?: "filename" | "path" | "frontmatter"; // How to generate slugs (default: "filename") 22 + slugField?: string; // Frontmatter field to use when slugSource is "frontmatter" (default: "slug") 23 + removeIndexFromSlug?: boolean; // Remove "/index" or "/_index" suffix from paths (default: false) 24 + textContentField?: string; // Frontmatter field to use for textContent instead of markdown body 25 } 26 27 export interface Credentials { ··· 45 frontmatter: PostFrontmatter; 46 content: string; 47 rawContent: string; 48 + rawFrontmatter: Record<string, unknown>; // For accessing custom fields like textContentField 49 } 50 51 export interface BlobRef {

History

5 rounds 1 comment
sign up or login to add to the discussion
7 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
chore: added linting and formatting
chore: linting updates
chore: refactored to use fallback approach if frontmatter.slugField is provided or not
expand 1 comment

For some reason tangled doesn't like rebasing, so going to close this out and open a fresh PR

closed without merging
6 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
chore: added linting and formatting
chore: linting updates
expand 0 comments
4 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
chore: updated comments
expand 0 comments
3 commits
expand
chore: resolved action items from issue #3
chore: adjusted tags to accept yaml multiline arrays for tags
chore: updated inject to handle new slug options
expand 0 comments
stevedylan.dev submitted #0
1 commit
expand
chore: resolved action items from issue #3
expand 0 comments