A harness, not a chat window.

01 · PremiseWhat the harness is for

A 7B model on a laptop is not a weaker Claude. It is a different failure surface: it invents tool names, it emits prose where JSON was demanded, it claims success it never earned. LocalNPC is a control structure built around those specific failures.

The shape is a library plus two binaries. localnpc (src/main.rs) is the CLI and daemon; npclocal (src/bin/npclocal.rs) launches the ratatui terminal console. Both drive the same AgentLoop. Nothing in the harness is model-specific: it speaks the OpenAI chat-completions wire format, so llama.cpp, vLLM, Ollama and LM Studio are all just a base_url.

Four commitments hold the design together, and each one is a mechanism rather than a prompt:

Gating

The model sees eight top-level tools. The forty-seven concrete tools — fs.read, shell.run, web.search, browser.click — are unreachable until their namespace is explicitly loaded.

Strictness

Each step must produce exactly one JSON object of the form {"name":…,"arguments":{…}}, validated against a compiled JSON Schema. Extra top-level keys are a parse error, not a warning.

Evidence

A run ends when required acceptance checks have passed. A model calling finish with open checks is rejected and re-prompted.

Replay

Every prompt, response, tool input, structured tool output, verification command and worker event is written to SQLite and JSONL as it happens.

02 · TopologyHow the crate is arranged

src/lib.rs re-exports twenty-two modules. They fall into four bands: entry points that parse arguments and layer configuration; the runtime core that owns the step loop; the state and model modules the loop reads and writes; and the surfaces — TUI, daemon, MCP, browser — that wrap or feed it.

The dependency direction is deliberately one-way. runtime/ depends on capabilities/, tools/, goal/, context/ and model/; none of those know the loop exists. tui/ and server/ sit on top and observe it through the same Event stream that replay consumes, which is why the terminal console and a recorded trace show identical timelines.

Counts are attribute counts from the working tree (#[test] and #[tokio::test]), not a pass/fail claim — the suite is run with cargo test and browser-dependent tests are #[ignore]d by default.

03 · First contactTwo commands that need no model

--self-test and doctor both exercise the scheduler without asking a model for anything. --self-test resolves the configured orchestrator and prints the decision. doctor goes one step further: it fetches a queue snapshot from the host, re-runs the scheduler with the observed busy slots, resolves the API credential and reports which environment variable or file it came from, then calls /models on the upstream.

The credential resolution is worth noting. resolve_api_key_with_source in src/app.rs:1269 returns both the value and its provenance, so doctor can print api_key_source= and tell you which of several fallback environment variables actually supplied the key — a common source of “why is it 401ing” on a machine with three half-configured providers.

$ cargo run -- --self-test
localnpc self-test ok
orchestrator=qwen3-coder-30b
host=workstation
worker_slots=2

$ cargo run -- doctor
model=qwen3-coder-30b
api_model=Qwen3-Coder-30B-A3B-Instruct
host=workstation
base_url=http://127.0.0.1:8000/v1
api_key_source=LOCALNPC_API_KEY
available_slots=4
worker_slots=2
queue_busy_slots=0
models_health=ok

$ cargo run -- models
qwen3-coder-30b host=workstation quality=8 speed=6 context=262144
qwen3-4b-instruct host=laptop quality=4 speed=9 context=32768

$ LOCALNPC_BASE_URL=http://127.0.0.1:8000/v1 \
    LOCALNPC_API_MODEL=your-model-name \
    cargo run -- run "Verify this Rust project"

Configuration is layered, then overridden

LocalNpcConfig::load takes the first file that exists from --config, then .localnpc/config.toml, then .localnpc.toml, then ~/.config/localnpc/config.toml, then a legacy .localnpc.json. Environment variables are applied on top, and only then does validate() run.

Validation is not decorative. It refuses a config whose default orchestrator is not in models, a model that references a missing host, a host with max_concurrency = 0, or a host with an empty base_url. The failure arrives at startup with the offending id in the message rather than as a confusing 404 twenty steps into a run.

The comment above default_model_max_tokens() is the kind of thing that only gets written after the bug: “A 512-token cap truncates any non-trivial fs.write mid-JSON, turning a self-inflicted budget limit into a parse_error.” The harness treats its own defaults as part of the failure surface.

04 · Beyond one boxThe daemon and the fleet router

localnpc serve starts an axum daemon on 127.0.0.1:8787 by default. It registers machines, discovers peers over Tailscale, launches and tunes local runtimes, and exposes forty-five HTTP routes — runs, events, tools, machines, runtimes, downloads, benchmarks and optimisation jobs.

The most interesting piece is src/server/router.rs, the fleet router. Its problem statement is stated plainly at the top of the file: without it, a client has to know which machine and port a model lives on. The router collapses that. A request names a model; the router resolves it to wherever it is actually served and proxies there.

The router is deliberately dumber than the harness. It does pure passthrough: no agent loop, no tool calls. That is what lets streaming, token usage and reasoning fields pass through untouched, and it is why the file draws a hard line between itself and the NPC-harness agent that also lives at /v1/chat/completions. It also never auto-serves a model — starting a runtime could disrupt a pinned GPU — so it will only route to something already up or to a configured upstream.

Registering the daemon in a client as a single OpenAI provider with base http://127.0.0.1:8787/fleet/v1 makes every fleet model appear under it. The proxied-completion ceiling is generous by design — ROUTE_TIMEOUT is 600 seconds — because long generations on a slow remote are the normal case, not the exception.