README.md

code-index #

Semantic search across local source repositories, running entirely on CPU.

Queries are written in plain language — "where do we decide whether a window stays on top", "the retry backoff for failed uploads" — and match on meaning rather than on the exact identifiers a grep would need. It complements grep rather than replacing it: grep wins when the exact string is known, this wins when it isn't.

Everything is local. No text leaves the machine.

Usage #

bin/code-index index                    # build or refresh (defaults to $CODE_INDEX_ROOT)
bin/code-index index --root /some/dir   # index a different tree
bin/code-index search 'how are stale sessions expired' -k 8
bin/code-index search 'ipc message routing' --repo my-app -q
bin/code-index search 'retry backoff' --json
bin/code-index status
bin/code-index stop                     # release the memory the server holds

--repo filters to one repository, --path to paths containing a substring, -q prints paths and scores without source text, and --json emits structured results for another program to consume.

Add bin/ to PATH, or call the script by its full path; it re-execs itself through uv so no virtualenv needs activating.

As an MCP server #

code-index-mcp exposes the index over stdio so an agent can search without shelling out. Register it once, for every session on the machine:

claude mcp add --scope user code-index <repo>/.venv/bin/code-index-mcp

Two tools:

  • search_code(query, limit, repo, path, paths_only) — ranked chunks with repository, path, and line range. Source text is capped per hit so a wide search cannot flood the caller's context.
  • index_status() — coverage, freshness, and whether the embedding server is up.

Indexing is deliberately not exposed as a tool: a first build takes hours, and an agent should not be able to start one by accident. Run code-index index instead.

Every session gets its own short-lived MCP process, but they all share one embedding server and one index. The server starts on the first search rather than at connection time, because loading the model takes long enough to trip a client's startup timeout.

How it works #

  • Discovery (files.py) walks each repository's git-tracked files, so .gitignore is honoured without reimplementing it, then drops binaries, lockfiles, minified output, and dependency or build directories. The exclusion list in EXCLUDED_SEGMENTS is deliberately conservative — a wrongly excluded file is invisible to search with no error, which is worse than a little noise.
  • Chunking (chunk.py) splits files into ~320-token line-aligned windows with a 64-token overlap, nudging boundaries onto the nearest function, class, or heading so a chunk tends to start at a definition. Each chunk is embedded with its file path prepended, which lets a query match a well-named file whose body never says the word.
  • Embedding (embed.py) calls a local llama-server running Qwen3-Embedding-0.6B. How that model has to be run — 1024 dimensions, last-token pooling, an instruction prefix on queries and none on documents — is looked up in embed.PROFILES by the model's filename. Each of those belongs to the model rather than to this deployment, and running a model with another model's settings degrades or destroys every result without raising anything.
  • Storage (store.py) keeps chunk metadata in SQLite and vectors in a memory-mapped float32 matrix. Search is a brute-force matrix multiply — at workspace scale that is well under a millisecond, so an approximate-nearest- neighbour index would add a dependency and a failure mode for no gain.

Re-running index re-embeds only files whose contents changed, matched by SHA-256. The index is written after each repository, so an interrupted run loses at most the repository in flight and re-running resumes from there.

Because the embedding server is shared machine-wide, it can vanish underneath a long-running index — another session stopping it, an out-of-memory kill. Requests retry, restarting the server if needed, so a lost server costs seconds rather than the whole run. Readiness is checked with the server's /health endpoint rather than by connecting to the port: llama-server binds its socket before the model has loaded and answers 503 until it is done.

Layout #

Path Contents
src/code_index/ the package
vendor/llama-*/ prebuilt llama.cpp binaries (not committed)
models/ the GGUF embedding model (not committed)
~/.local/share/code-index/ SQLite index, vector matrix, server log

Both the binaries and the model are downloaded artifacts rather than build outputs; setup.sh fetches them.

Environment #

Variable Purpose
CODE_INDEX_PORT embedding server port (default 8177)
CODE_INDEX_MODEL path to a different GGUF embedding model, which needs an entry in embed.PROFILES
CODE_INDEX_LLAMA_BIN directory holding llama-server
CODE_INDEX_DATA where the index is stored
CODE_INDEX_ROOT directory holding the repositories to index (default ~/workspace)
CODE_INDEX_EVAL_QUERIES query set for eval/score.py (default eval/queries.local.jsonl)

tracks.json points CODE_INDEX_DATA at a directory inside each git worktree, so a build started while experimenting cannot overwrite the shared index that every session searches. The embedding server is deliberately not namespaced: it holds no index state and is meant to be shared. It is also the largest single process on the machine while it runs — see the memory figure below.

Measured on this machine #

16 vCPU AMD EPYC Rome, no GPU, 66.7 GB/s memory bandwidth.

Embedding throughput ~1.4 chunks/s
Full index of ~2,400 files / 37k chunks ~7.4 hours
Refresh after a normal day's edits seconds to a minute
Vector matrix at 37k chunks 154 MB
Query latency ~120 ms, nearly all of it embedding the query
Server resident memory ~4.8 GB

Throughput and the full-index time are from eval/model-comparison.md, measured on a machine that was not idle. Resident memory settles after a few hundred chunks rather than at model load, and is mostly key/value cache: eval/batch-size.md measures it at 112 KB per token of context, which is why BATCH_SIZE is 8 and the server's 8 GiB prompt cache is switched off. It was ~600 MB under the previous model, which as an encoder keeps no key/value cache at all. code-index stop releases it.

The first index is slow because CPU embedding is compute-bound; every run after it is incremental. Leaving the server running costs memory but removes a ~10 s model load from the first query.

Choosing a different model #

Qwen3-Embedding-0.6B is the default because it retrieves substantially better on this workspace's own queries: unscoped top-1 rose from 40% to 85% and top-5 from 70% to 100% against nomic-embed-text-v1.5, at 1.7x slower indexing — about 7.4 hours for a full rebuild rather than 4.3. eval/model-comparison.md records the measurement and what the corpus was; the query set itself is a local file, since a useful one names real files in real repositories — see eval/README.md.

To switch to another model, add an entry to PROFILES in embed.py giving its dimensions, pooling strategy, and task prefixes, point CODE_INDEX_MODEL at the GGUF, and rebuild with --force. A model with no entry is refused rather than run on another model's settings: pooling in particular is silent when it is wrong — llama-server accepts --pooling mean against a decoder-only model and returns vectors of the right shape that mean nothing. The rebuild is not optional either; vectors from two models cannot be compared, which is why a changed model or dimension re-embeds everything rather than reusing anything.