retrieval

The router is a paragraph of prose

There is no classifier, no embedding of the query against tool descriptions, no orchestration layer. The routing policy is eight lines of text pasted into five tool descriptions, and the model is expected to read it.

This is the most consequential design decision in the repository, and it is also the least code. A constant named TOOL_ROUTER_NOTE is defined once and interpolated into the description of files_search, wiki_index, files_neighbors, wiki_read_all and files_read. Whichever of those five tools the model is looking at, it sees the whole map — including the tools it should have called instead.

src/tools/files.ts:17-24TOOL_ROUTER_NOTE, quoted in full
Router (for any agent, cloud or local):
  • Don't know what's in the wiki yet? → wiki_index (cheap)
  • Specific question, want best snippet? → files_search (mode=vector default; "keyword" for exact)
  • Need a page + everything connected to it? → files_neighbors (Zoom-Out)
  • Know the path? → files_read
  • Cloud agent, big context, audit-style "everything about X"? → wiki_read_all
  • Personal memory / "what did we decide / when"? → memory_recall
  • "What connects A to B?" (graph) → graph_query / graph_path

Note what each rule keys on. Not the topic of the question — the shape of it, and the state of the agent's own knowledge. “Don't know what's in the wiki yet” is a fact about the conversation, not about the corpus.

The retrieval router as a decision tree A question enters at the top and is classified by shape. Six outcomes are shown with their cost bands: wiki_index for orientation, files_read for a known path, files_search in keyword, vector or deep mode for a snippet, files_neighbors for a page and its connections, graph_query for multi-hop connection questions, and wiki_read_all for whole-corpus audit questions, which is gated on the agent having a large context window. A cost axis runs down the left from cheap to expensive, and an inset shows the default two-step loop. cheap costly a question arrives what shape is it? wiki_index “what pages even exist?” ≈500 tokens · no arguments files_read “I already know the path” one file · head -c max_bytes wiki_read_all “everything about X”, audit style ≈60–100K tokens of the whole corpus gated: cloud agents with large context only files_search “find me the best snippet” keyword → BM25 · exact ~50 ms vector · the default ~200 ms deep → expand + rerank 1–2 s files_neighbors “that page and all it touches” the Zoom-Out step · ≈2–10K tokens graph_query / graph_path “what connects A to B?” · multi-hop token-budgeted traversal, default 2,000 the default loop, for any agent files_search — localise the evidence files_neighbors — expand the context answer — escalate only if that failed
The router. Five of the six outcomes are cheap. The expensive one — reading the entire corpus — is the only branch with a precondition attached, and the precondition is a property of the client, not of the question. Latency figures are those stated in the tool descriptions at src/tools/files.ts:89 and :99.

Search is a subprocess, not a service

files_search does not speak HTTP to a search server. It maps the requested mode to a subcommand and executes a command-line binary, once per call. The comment in the source is unusually direct about why:

Backed by QMD CLI (shell-out per call — bypasses upstream MCP transport bug).

src/tools/files.ts:89 — files_search description

A downstream MCP server for the same index exists in the federation configuration and is disabled, with a recorded reason: its HTTP transport returns an error page rather than a valid handshake. Rather than wait for a fix, the tool drops one layer down to the CLI that server wraps. The mode names in the MCP schema and the subcommand names on disk are therefore different vocabularies, mapped explicitly:

Mode mapping and timeouts — src/tools/files.ts:104-117
MCP mode CLI subcommand Method Timeout Stated latency
keywordsearchBM25, exact match15 s~50 ms
vector defaultvsearchSemantic embedding search30 s~200 ms
deepqueryQuery expansion, then LLM rerank120 s1–2 s

The output cap is 50 MB per invocation and results default to ten, capped at fifty. Both the query and the mode are recorded in the audit ledger — the query truncated to its first hundred characters.

Degrading in three steps

The failure design is the most carefully written part of the module. A failed search does not raise; it falls back to a strictly cheaper method, twice, before giving up. The chain is asymmetric on purpose: deep degrades to vector, anything else degrades to keyword, and keyword is the floor.

src/tools/files.ts:118-131the fallback ladder
try {
  text = await callQmd(cliMode, timeouts[cliMode] || 30_000);
} catch (err: any) {
  const fallbackMode = (cliMode === 'query') ? 'vsearch' : 'search';
  console.warn('[files_search] ' + cliMode + ' failed: ' + err.message + ', falling back to ' + fallbackMode);
  try {
    text = await callQmd(fallbackMode, timeouts[fallbackMode] || 30_000);
  } catch {
    console.warn('[files_search] ' + fallbackMode + ' also failed, last resort keyword');
    text = await callQmd('search', 15_000);
  }
}

The degradation is invisible to the caller. The model receives results and no indication that it got BM25 when it asked for a reranked semantic search — only the server's stderr records the downgrade. Worth knowing when reasoning about why an answer was thin.

One more piece of defensive plumbing sits immediately after: the CLI's stdout is not assumed to be clean JSON. The handler slices from the first [ to the last ], discarding any banner or progress text the binary may have emitted around the payload.

the subprocess layer, as files_search drives it
# mode="keyword" → 15s timeout
$ qmd search "transport handshake failure" -n 10 --json
[{"path": "…", "score": 11.4, "snippet": "…"}, …]

# mode="vector" (default) → 30s timeout
$ qmd vsearch "why did we choose that transport" -n 10 --json
[{"path": "…", "score": 0.83, "snippet": "…"}, …]

# mode="deep" → 120s timeout; if it throws, the ladder engages
$ qmd query "everything about the transport decision" -n 10 --json
error: rerank backend unreachable
[files_search] query failed: rerank backend unreachable, falling back to vsearch
[{"path": "…", "score": 0.79, "snippet": "…"}, …]

# and the graph tools, driven the same way — one process per call
$ python -m graphify query "what connects A to B" --graph <wiki>/graphify-out/graph.json --budget 4000
$ python -m graphify path "<node A>" "<node B>" --graph <wiki>/graphify-out/graph.json

Argument vectors are exact, from src/tools/files.ts:111 and src/tools/graph.ts:45-47. Binary paths, the wiki location and result payloads are replaced with placeholders.

The lifecycle of one question

Put the pieces in order and a single retrieval looks like this. Note where the boundaries are: authentication happens once per HTTP request, the MCP server object is built and destroyed around it, and the audit row is written before the response is serialised.

Lifecycle of a single retrieval call A sequence diagram with five lifelines: the agent, the HTTP layer, the tool handler, backends, and the audit ledger. The agent posts a tools/call request; the HTTP layer authenticates the request and constructs a per-request MCP server; the handler validates arguments with Zod, spawns a search subprocess, trims the JSON payload, writes an audit row and returns text content. The agent then issues a second call to files_neighbors, which reads the corpus from disk. Finally the response closes and both the transport and the MCP server are torn down. agent express + auth tool handler backends audit.db POST /mcp — tools/call files_search {query, mode:"vector"} Authorization: Bearer … verified before the tool layer is constructed · 401 otherwise new McpServer(…) + registerAllTools(…) — per request Zod validates: query non-empty, limit 1–50, mode ∈ {deep,keyword,vector} mode → subcommand: vector ⇒ vsearch execFile(qmd, ["vsearch", query, "-n", "10", "--json"]) one process per call 50 MB stdout ceiling 30 s timeout stdout — possibly with a banner around the JSON slice from first “[” to last “]” — files.ts:132-134 logCall("files_search", {query: query.slice(0,100), mode}, "success", ms) arguments stored only as sha256(args)[0:16] { isError: false, content: [{ type: "text", text }] } the Zoom-Out step — the agent now knows which page it wants tools/call files_neighbors {slug} walkWiki() + readFile on every page — no subprocess, no index target page + outgoing + inbound, capped at max_bytes res.on("close") → transport.close(); mcpServer.close() — server.ts:120-123
One question, two calls. The search localises evidence; the neighbourhood call reconstructs the surrounding structure. The source names this the MemCoT Zoom-Out pattern and prescribes it as the default loop for any agent (src/tools/files.ts:178-180, src/server.ts:100-102).

The client's context window is part of the API

Most tool descriptions describe what a tool does. Two of these describe who should call it. The distinction being drawn is not capability but economics — a hosted frontier model paying per token behaves differently from a quantised local model whose KV cache is a fixed budget.

Cloud agent, large context

  • wiki_read_all is permitted for audit-style questions — “give me everything about X”.
  • Cost is roughly 60–100K tokens at the wiki's stated size.
  • The default 1 MB cap is a safety valve, not a target.

Local agent, 128K context

  • wiki_read_all is explicitly discouraged: it “fills KV cache, slows decode ~30–45%”.
  • Use files_search then files_neighbors instead.
  • files_neighbors is documented as staying “well within KV cache budget”.

A claim, not a measurement

The 30–45% decode slowdown and the 60–100K token estimate appear as assertions in the tool descriptions at src/tools/files.ts:302-306. There is no benchmark in the repository that produces them. They are reproduced here because they are what the model is told — they shape behaviour whether or not they are precise.

Escalation, not selection

Read the router as a ladder rather than a menu and the design intent becomes clear. The agent is expected to start at the bottom and climb only when the cheap rung fails to answer. Every rung returns something the next rung can use: an index yields paths, a search yields a slug, a slug yields a neighbourhood, a neighbourhood yields the names that make a graph query answerable.

Default loop for any wiki question: files_search to find the page → files_neighbors to expand context → answer. Only escalate to wiki_read_all on cloud agents for true audit questions.

src/server.ts:100-102 — client instructions, condensed

Queries shown in the terminal are illustrative; no real query, result or wiki path appears on this site. Binary locations and the wiki root are replaced with placeholders.