llama-cpp-bonsai by okami.mom

llama.cpp fork for Ternary Bonsai 2 27B: MTP speculation, suffix cache reuse (SCR), q8 KV, two-slot fairness, fused DeltaNet kernels

Branches

Name

Commit

Message

Date

main Default

cache: conversations are stored against their learned prefix

1w ago

moe-cache: LLAMA_MOE_CACHE_MAP/LLAMA_MOE_CACHE_SPLIT place a layer cache on another GPU; moe-trace tools for routing traces

1w ago

main Default

8906fd8c · 1w ago

87e1b429 · 1w ago