MLF Integration Test Suite#

This directory contains workspace-level integration tests that span multiple crates. Unlike per-crate unit tests, these verify end-to-end functionality across the entire MLF toolchain.

Test Organization#

By Scope#

tests/
├── codegen/           # mlf-codegen: lexicon generation
│   ├── lexicon/       # JSON lexicon output
│   ├── typescript/    # mlf-codegen-typescript
│   ├── rust/          # mlf-codegen-rust
│   └── go/            # mlf-codegen-go
├── cli/               # mlf-cli: command-line interface
├── diagnostics/       # mlf-diagnostics: error formatting
├── workspace/         # Multi-file resolution, .mlf dirs
└── real_world/        # Full lexicon suites (bsky, place.stream)

By Crate#

Single-crate tests (in crate's own tests/ dir):

• mlf-lang/tests/ - Language semantics (parsing, validation) - 17 tests ✅
• mlf-codegen/tests/ - Core codegen (if any crate-specific)
• mlf-cli/tests/ - CLI-specific unit tests

Multi-crate tests (in workspace tests/ dir):

• Full end-to-end workflows
• Integration between crates
• Real-world scenarios

Current Status#

✅ Implemented#

• mlf-lang/tests/lang/ - 21 tests for parsing and validation
• tests/codegen/lexicon/ - 10 tests for lexicon generation
• tests/real_world_roundtrip - Round-trip test (JSON → MLF → JSON) with real lexicons

🚧 Planned#

Codegen Tests#

• codegen/lexicon/ - MLF → JSON lexicon generation

  • Implicit main resolution
  • Inline type expansion
  • Annotation handling (@key, @encoding)
  • Reference resolution (local vs imported)

• codegen/typescript/ - TypeScript code generation

  • Basic records → interfaces
  • Union types
  • Optional/required fields
  • Import statements

• codegen/rust/ - Rust code generation

  • Structs with serde
  • Enums
  • Lifetimes
  • Option for optional fields

• codegen/go/ - Go code generation

  • Structs with json tags
  • Pointers for optional fields

CLI Tests#

• cli/check - mlf check command
• cli/generate - mlf generate lexicon/code
• cli/validate - mlf validate with schemas
• cli/fetch - mlf fetch from network
• cli/init - mlf init project setup

Diagnostics Tests#

• diagnostics/error_quality - Error message formatting
• diagnostics/suggestions - Import suggestions, typo corrections
• diagnostics/multiple_errors - Batch error reporting

Workspace Tests#

• workspace/local_mlf - .mlf/lexicons/ resolution
• workspace/home_mlf - ~/.mlf/lexicons/ resolution
• workspace/precedence - Resolution order (local > home > std)
• workspace/sibling_files - Multi-file modules

Real-World Tests ✅#

• real_world/ - Tests using real lexicons from production networks
  • roundtrip - Round-trip test: JSON → MLF → JSON
    • Fetches real lexicons (app.bsky., net.anisota., place.stream., pub.leaflet.)
    • Validates accurate conversion both ways
    • Writes diff files to real_world/roundtrip/diffs/ (gitignored)
    • Run with: just test-real-world
    • See real_world/README.md for details

Running Tests#

We use just for test execution. Install with: cargo install just

Quick Start#

# Run all tests (recommended)
just test

# Run specific test category
just test-lang        # Language tests (17 tests)
just test-codegen     # Codegen tests (4 tests)
just test-validation  # Validation tests (12 tests)

# Network-dependent tests (run explicitly)
just test-real-world  # Round-trip test: fetch real lexicons, convert MLF→JSON, verify

# Other useful commands
just test-all         # All workspace tests (includes unit tests)
just test-verbose     # Tests with verbose output
just test-stats       # Show test statistics
just test-list        # List all test directories

Using Cargo Directly#

# All tests (excluding problematic packages)
cargo test --workspace --exclude tree-sitter-mlf --exclude mlf-wasm

# Specific test suite
cargo test -p mlf-lang --test integration_test
cargo test -p mlf-integration-tests --test codegen_integration

# With output
cargo test -p mlf-lang --test integration_test -- --nocapture

Test Infrastructure#

Rust-based Test Runner#

All integration tests use automatic test discovery:

• Test cases in directories with input.mlf + expected.json
• Test runner walks directories and executes each test
• Clear pass/fail reporting with ✓/✗ indicators
• Used for: lang, codegen, diagnostics, workspace tests

Just Recipes#

The justfile at the workspace root provides convenient test execution:

• No shell scripts needed
• Consistent interface across test types
• Proper exclusion of problematic packages
• Easy to extend for new test categories

Writing Multi-Crate Tests#

Example: Codegen Test#

tests/codegen/lexicon/basic_record/
├── input.mlf              # MLF source
├── expected.json          # Expected lexicon output
└── test_config.toml       # Test metadata

Test runner:

// tests/codegen_integration.rs
use mlf_lang::{parser::parse_lexicon, Workspace};
use mlf_codegen::LexiconGenerator;

#[test]
fn codegen_tests() {
    for test_dir in discover_tests("tests/codegen") {
        let mlf = read_mlf(test_dir);
        let expected = read_expected(test_dir);

        // Parse (mlf-lang)
        let lexicon = parse_lexicon(&mlf).unwrap();

        // Generate (mlf-codegen)
        let output = LexiconGenerator::generate(lexicon).unwrap();

        // Compare
        assert_eq!(output, expected);
    }
}

Example: CLI Test#

CLI tests follow the same pattern as codegen tests - Rust-based test runner with automatic discovery:

tests/cli/check_command/
├── input.mlf              # MLF source to check
├── expected.json          # Expected result (status + output)
└── test_config.toml       # CLI args to pass

Test runner:

// tests/cli_integration.rs
use std::process::Command;

#[test]
fn cli_tests() {
    for test_dir in discover_tests("tests/cli") {
        let input = read_mlf(test_dir);
        let expected = read_expected(test_dir);

        // Run CLI command
        let output = Command::new("mlf")
            .arg("check")
            .arg(input_path)
            .output()
            .unwrap();

        // Verify exit code and output
        assert_eq!(output.status.success(), expected.success);
        assert_eq!(String::from_utf8_lossy(&output.stdout), expected.stdout);
    }
}

Test File Conventions#

Required Files#

Each test directory must contain:

1. test.toml - Test configuration (REQUIRED for multi-file tests)

   [test]
   name = "namespace_alias"
   description = "Test namespace aliasing"
   namespace = "com.example.thing"  # For codegen tests
   
   # For multi-file lang tests, specify namespace per file
   [modules]
   "test.mlf" = "com.example.thing"
   "defs.mlf" = "com.example.defs"
   

2. MLF source files

   • test.mlf - Main test file (lang tests)
   • input.mlf - Main input file (codegen tests)
   • Additional .mlf files as needed (specified in test.toml)

3. Expected results

   • expected.json - Expected output (status, defs, or errors)
   • For codegen: Full lexicon JSON output
   • For lang: {"status": "success"} or {"status": "error", "errors": [...]}

Example Test Structures#

Lang test (single file):

tests/lang/constraints/known_values/
├── test.toml          # Optional: auto-derives namespace if missing
├── test.mlf
└── expected.json

Lang test (multiple files):

tests/lang/namespace_imports/namespace_alias/
├── test.toml          # Required: maps filenames to namespaces
├── test.mlf
├── defs.mlf
└── expected.json

Codegen test:

tests/codegen/lexicon/basic_record/
├── test.toml          # Required: specifies namespace
├── input.mlf
└── expected.json      # Full lexicon JSON

Benefits of Workspace-Level Tests#

1. Cross-crate integration - Test full workflows (parse → validate → codegen)
2. Real-world scenarios - Actual user workflows, not isolated units
3. Regression prevention - Catch breaking changes across crate boundaries
4. Living documentation - Show how crates work together
5. CI/CD validation - One command tests entire toolchain

Migration Plan#

1. ✅ Keep current mlf-lang tests where they are
2. 🚧 Create workspace-level test runners
3. 🚧 Add codegen tests (require mlf-codegen)
4. 🚧 Add CLI tests (require mlf-cli)
5. 🚧 Add real-world tests (require all crates)

See Also#

• mlf-lang/tests/README.md - Language-specific test docs
• mlf-cli/tests/README.md - CLI test docs
• CLAUDE.md - Project overview and architecture