teal.fm
1# Lexicon CLI Tool
2
3A unified command-line tool for managing AT Protocol lexicons across TypeScript and Rust codegen in the Teal monorepo.
4
5## Features
6
7- **Unified Generation**: Generate both TypeScript and Rust types from lexicons with a single command
8- **Auto-installation**: Automatically installs required dependencies (esquema-cli) when needed
9- **File Watching**: Watch lexicon files for changes and auto-regenerate types
10- **Cross-language Validation**: Validate consistency between TypeScript and Rust generated types
11- **Diff Analysis**: Show impact of lexicon changes on generated code
12
13## Installation
14
15The tool is automatically available in the workspace. Install dependencies:
16
17```bash
18pnpm install
19```
20
21## Commands
22
23### Generate Types
24
25Generate both TypeScript and Rust types from lexicons:
26
27```bash
28# Generate all types
29pnpm lex:gen
30
31# Generate only TypeScript types
32lex gen --ts-only
33
34# Generate only Rust types
35lex gen --rust-only
36
37# Force regeneration even if no changes detected
38lex gen --force
39```
40
41### Watch for Changes
42
43Automatically regenerate types when lexicon files change:
44
45```bash
46# Watch all lexicon files
47pnpm lex:watch
48
49# Watch only for TypeScript generation
50lex watch --ts-only
51
52# Watch only for Rust generation
53lex watch --rust-only
54```
55
56### Validate Types
57
58Validate that generated types are consistent and up-to-date:
59
60```bash
61pnpm lex:validate
62```
63
64### Show Changes
65
66Display lexicon and generated type changes:
67
68```bash
69# Show changes since last commit
70pnpm lex:diff
71
72# Show changes since specific commit
73lex diff HEAD~3
74```
75
76## How It Works
77
78### TypeScript Generation
79- Uses `@atproto/lex-cli` to generate TypeScript types
80- Sources lexicons from `packages/lexicons/real/`
81- Outputs to `packages/lexicons/src/types/`
82
83### Rust Generation
84- Uses `esquema-cli` (forked from atrium-codegen)
85- Sources lexicons from `services/types/lexicons/`
86- Outputs to `services/types/src/`
87- Auto-installs esquema-cli if not present
88
89### File Watching
90- Monitors both lexicon source directories
91- Debounces changes to avoid multiple regenerations
92- Shows clear feedback about what changed
93
94## Integration with Build System
95
96The tool integrates with the existing Turbo build pipeline:
97
98- `lex:gen-server` - Legacy TypeScript-only generation (kept for compatibility)
99- `lex:gen` - New unified generation command
100- `lex:watch` - File watching for development
101- `lex:validate` - Type validation
102
103## Lexicon Sources
104
105### Teal Lexicons (`fm.teal.alpha.*`)
106- **Actor**: Profile management, status tracking, search
107- **Feed**: Music play tracking, artist/track data
108- **Stats**: Top artists, releases, user statistics
109
110### External Protocol Support
111- **Bluesky** (`app.bsky.*`): Full AT Protocol compatibility
112- **Statusphere** (`xyz.statusphere.*`): Status sharing
113- **AT Protocol Core** (`com.atproto.*`): Authentication, repositories, admin
114
115## Developer Workflow
116
1171. **Making Lexicon Changes**: Edit files in `packages/lexicons/real/` or `services/types/lexicons/`
1182. **Regenerate Types**: Run `pnpm lex:gen` or use `pnpm lex:watch` during development
1193. **Validate Changes**: Run `pnpm lex:validate` to check consistency
1204. **Review Impact**: Use `pnpm lex:diff` to see what changed
121
122## Troubleshooting
123
124### esquema-cli Installation Issues
125If automatic installation fails:
126```bash
127cargo install esquema-cli --git https://github.com/fatfingers23/esquema.git
128```
129
130### Type Generation Failures
131- Ensure lexicon JSON files are valid
132- Check that both TypeScript and Rust lexicon directories exist
133- Verify file permissions for output directories
134
135### Watch Mode Issues
136- File watching uses `chokidar` for cross-platform compatibility
137- If changes aren't detected, try restarting the watcher
138- Large numbers of files may cause performance issues
139
140## Configuration
141
142The tool automatically detects the workspace structure using:
143- `pnpm-workspace.yaml` for workspace root detection
144- Package structure for TypeScript and Rust paths
145- Git for diff operations
146
147No additional configuration is required.