- Python 100%
Tyro CLI utilities #
A set of utilities for building CLI interfaces with Tyro.
Motivated by https://github.com/brentyi/tyro/issues/428. Many of the utilities are inspired by Hydra.
Core utilities (tyroutils.core) #
Lazy fields #
Sometimes a field's default can only be computed after the user has made a subcommand selection (e.g. the learning rate schedule depends on the optimizer chosen). Mark such fields with LAZY_MISSING and type them with Lazy[T]:
from dataclasses import dataclass
from tyroutils.core import LAZY_MISSING, Lazy, is_lazy
@dataclass
class TrainConfig:
optimizer: str = "adam"
learning_rate: Lazy[float] = LAZY_MISSING # filled in later
cfg = TrainConfig()
print(is_lazy(cfg.learning_rate)) # True
Tyro presents Lazy[float] as float in the CLI help, so users can always override the computed default from the command line.
Two-pass CLI parsing with specialize_cli #
In tyro, union-typed nested dataclass fields become subcommands. A common need is to set defaults on other fields based on that selection, e.g., the model's num_classes depends on which dataset was chosen.
specialize_cli handles this with a two-pass parse: it discovers the user's subcommand selections first, runs a specializer to compute derived defaults, then does the full parse with those defaults in place (so users can still override any of them with flags):
from dataclasses import dataclass, field
from typing import Union, Annotated
import tyro
from tyroutils.core import LAZY_MISSING, Lazy, specialize_cli
@dataclass
class MNISTConfig:
num_classes: int = 10
@dataclass
class CIFAR10Config:
num_classes: int = 10
@dataclass
class MLPConfig:
num_classes: Lazy[int] = LAZY_MISSING # filled in from dataset choice
hidden_dim: int = 256
@dataclass
class Config:
data: Union[MNISTConfig, CIFAR10Config] = field(default_factory=MNISTConfig)
model: MLPConfig = field(default_factory=MLPConfig)
def specialize(skeleton: Config) -> Config:
skeleton.model.num_classes = skeleton.data.num_classes
return skeleton
cfg = specialize_cli(Config, specialize)
Then at the command line:
# select dataset subcommand; model.num_classes is filled automatically
python train.py data:cifar-10-config
# user can still override the specialized default
python train.py data:cifar-10-config --model.num_classes 5
How it works:
- Pass 1 filters to bare tokens containing
:(subcommand selectors likedata:cifar-10-config) and parses just those to build a skeleton config with class-level defaults. - The specializer updates fields on the skeleton using the selected types.
- Pass 2 runs the full
tyro.cli()parse with the specialized skeleton as defaults, so user flags always win.
Tagged union serialization #
auto_configure_tagged_unions registers cattrs hooks that disambiguate Union types using a "type" discriminator key. TaggedSerializable wraps this into a convenient mixin:
from dataclasses import dataclass
from typing import Union
from tyroutils.core import TaggedSerializable
@dataclass
class Adam:
lr: float = 1e-3
@dataclass
class SGD:
lr: float = 1e-2
momentum: float = 0.9
@dataclass
class Config(TaggedSerializable):
optimizer: Union[Adam, SGD] = Adam()
cfg = Config(optimizer=SGD())
data = cfg.unstructure()
# {"optimizer": {"type": "SGD", "lr": 0.01, "momentum": 0.9}}
restored = Config.structure(data)
assert isinstance(restored.optimizer, SGD)
Lazy fields round-trip as the string "LAZY_MISSING" and are restored to the sentinel on .structure().
Useful templating (tyroutils.templates) #
BaseConfig provides automatic timestamped run directory management:
from dataclasses import dataclass
from tyroutils.templates import BaseConfig
import tyro
@dataclass
class Config(BaseConfig):
epochs: int = 10
lr: float = 1e-3
cfg = tyro.cli(Config)
cfg.change_dirs(output_dir="runs")
# cwd is now: runs/<name>/YYYY-MM-DD/HH-MM-SS/
# original working directory is still accessible
print(cfg.original_cwd)
Pass a suffix to annotate the run directory (e.g. with hyperparameters). Special characters are sanitized automatically:
cfg.change_dirs(suffix=f"lr={cfg.lr}", output_dir="runs")
# runs/experiment_lr=0.001/YYYY-MM-DD/HH-MM-SS/
Flax model configs (tyroutils.flax) #
build_model_cfg dynamically generates a dataclass config type from a Flax NNX module's __init__ signature. Required parameters become Lazy fields; parameters with defaults keep their defaults:
from flax import nnx
from tyroutils.flax import build_model_cfg
class MLP(nnx.Module):
def __init__(self, features: int, hidden: int = 256, *, rngs: nnx.Rngs):
...
MLPConfig = build_model_cfg(MLP)
# equivalent to:
# @dataclass
# class MLPConfig:
# features: Lazy[int] = LAZY_MISSING
# hidden: int = 256
cfg = MLPConfig(features=128)
model = cfg.build(rngs=nnx.Rngs(0))
The generated config class also works with specialize_cli and TaggedSerializable since its fields follow the same conventions.