Wiring codex to a Local Qwen — ollama, vLLM, and the developer role

OpenAI's codex CLI defaults to OpenAI's cloud, but the same binary happily talks to a local ollama or vLLM once two knobs are set right. On ollama the entire setup is a single config.toml. On vLLM there is exactly one thing that breaks — the developer role — and it can be fixed either at the server (chat template patch) or at the edge (a tiny proxy). This post walks the shortest working paths and shows why the vLLM breakage happens.

---

The wiring in one picture

codex sends /v1/responses requests. Any backend that serves that endpoint can host it:

[ollama]       codex ─(/v1/responses)──────────────────▶ ollama:11434
[vLLM direct]  codex ─(/v1/responses)──────────────────▶ vLLM:8000      (with chat_template patch)
[vLLM proxy]   codex ─(/v1/responses)─▶ proxy.py ─(fix)─▶ vLLM:8000      (when the server is shared)

Two things have to hold on every path:

1. A custom provider with wire_api = "responses". codex 0.137+ forces the Responses API, and 0.142+ will fall back to a WebSocket transport on the default provider if you only override openai_base_url. Declare the provider explicitly.
2. An isolated CODEX_HOME. Point codex at the repo's .codex/ instead of ~/.codex so personal hooks, trust files, and logs stay out of the experiment.

Everything else is a variation on these two.

Five-minute quick start with ollama

ollama needs no proxy. It accepts the developer role that codex sends and serves /v1/responses directly.

Prep: codex --version (0.137+) and one tool-calling model, e.g. ollama pull gemma4:12b.

Put the model name in .codex/config.toml:

model = "gemma4:12b"
model_provider = "local_ollama"

[model_providers.local_ollama]
name = "Local Ollama"
base_url = "http://localhost:11434/v1"
wire_api = "responses"

Then run codex with CODEX_HOME pointing at that isolated config:

CODEX_HOME="$PWD/.codex" codex exec \
  --skip-git-repo-check --ephemeral \
  --dangerously-bypass-approvals-and-sandbox \
  "Reply with exactly the single word: PONG"

If PONG comes back, wiring is good. Swap in a real coding prompt with -C /tmp/work and the model starts writing files.

One caveat: model size matters. A qwen3:0.6b is fine for smoke-testing the wire, but real coding needs a tool-calling model in the gemma4:12b class or above.

Even shorter — ollama's built-in codex launcher

Recent ollama ships an agent launcher. Running ollama with no subcommand opens a menu of coding agents — OpenCode, Hermes, Codex, Cline, Qwen Code, and more. Selecting Launch Codex (gemma4:12b) spawns codex 0.142.5 already wired to an ollama-served model, no config.toml edit required.

What the zero-config path costs:

• User-level config, no isolation. The launcher does not set CODEX_HOME. codex reads ~/.codex/config.toml, your global hooks, and your personal trust store. If a project-local .codex/config.toml is present in the launched cwd, codex quietly drops its provider keys and warns:

  Ignored unsupported project-local config keys in <cwd>/.codex/config.toml:
    model_provider, model_providers.
  If you want these settings to apply, manually set them in your user-level config.toml.

  Provider selection is a user-level decision on this path — project-local .codex/ from the previous section stops being the source of truth. That is a deliberate boundary: a project cannot silently repoint codex at a different backend just by shipping a config file.

  

• The rewrite is real, and codex-specific. In one measured run the launcher replaced ~/.codex/config.toml with a template pointing at a non-existent tag (gemma4:12b) and wire_api = "responses" — which does not match ollama's chat-completions shape. The next codex call from an unrelated project hung with no output. The hazard is codex-schema-specific: codex keeps its active provider as a single top-level model_provider = key, so a file-level template swap flips the live route. Peer agents whose config separates registration from activation were left alone in the same session — for example Claude Code's ~/.claude.json uses a providerProfiles[] list plus a separate activeProviderProfileId pointer, and the launcher added Ollama there as an inactive profile only. Snapshot the good config before running the launcher: cp ~/.codex/config.toml ~/.codex/config.openai.toml, restore with cp if it swaps.

• YOLO mode by default. The session opens with approvals and sandbox off. Fine on a personal laptop, less fine on shared machines.

• ollama only. The launcher targets ollama's own runtime. For vLLM the wiring below still applies.

Pick the launcher when the use case is one laptop, one ollama model, and no need to keep experiments apart. Keep the isolated CODEX_HOME flow when you want per-project sandboxes, side-by-side model comparisons, or the vLLM/proxy path.

vLLM — two ways past the developer role

Point codex at a vanilla vLLM and the first request dies with 400 "Unexpected message role." The role in question is developer, which codex uses for system-level instructions. Two clean fixes:

A. Patch the chat template (server-owned vLLM). If you control the server, add a developer branch to the Jinja template and pass it via --chat-template. This is the root-cause fix, no proxy on the path.

{%- elif message.role == "developer" %}
    {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}

B. Front the server with a small proxy (shared vLLM). When the vLLM instance is shared and you cannot restart it, sit an edge proxy in front and rewrite developer items into instructions (the leading system). The rest of the request is forwarded verbatim.

# Terminal A — the proxy
python proxy.py --vllm http://<host>:8000/v1 \
  --model Qwen/Qwen3.6-35B-A3B-FP8 --port 8731

# Terminal B — codex against the proxy
CODEX_HOME="$PWD/.codex" codex exec \
  -c model_providers.local_vllm.base_url="http://localhost:8731/v1" \
  -c model_providers.local_vllm.wire_api="responses" \
  -c model_provider="local_vllm" \
  -m Qwen/Qwen3.6-35B-A3B-FP8 \
  --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox \
  "Summarize the layout of this repo"

The proxy's job is deliberately small — pass every field of /v1/responses straight through, only touch the developer items, inject model into the body if codex 0.142+ left it blank. Usage counts, tool calls, and SSE streams all come from vLLM unchanged.

The one-shot runner run_codex.py wraps both the proxy and the codex call:

python run_codex.py \
  --vllm http://<host>:8000/v1 \
  --model Qwen/Qwen3.6-35B-A3B-FP8 \
  --cwd . --prompt "Summarize the layout of this repo" --mode cli

A healthy run ends with [turns=N errors=0] and codex's final answer.

Why vLLM rejects the developer role

The rejection does not come from vLLM's Python code. It comes from the model's Jinja chat template — the same file that decides how system, user, assistant, and tool become tokens:

{%- if message.role == "system" %} ...
{%- elif message.role == "user" %} ...
{%- elif message.role == "assistant" %} ...
{%- elif message.role == "tool" %} ...
{%- else %}{{- raise_exception('Unexpected message role.') }}

The 400 body ("Unexpected message role.", period included) matches this raise_exception string exactly. Because the failure lives in the template, both fixes work at the right layer: A teaches the template a new branch; B re-labels the item before it ever reaches the template. Existing traffic (system / user / assistant / tool) is byte-identical either way — the patched template only diverges on the developer branch, so shared workloads are unaffected.

If you can upgrade to a recent vLLM, the newer chat_utils also recognises developer natively, which is another server-side path.

SDK mode (optional)

The Python openai-codex SDK gets wired the same way — a custom model_providers entry in the config dict pointing at the proxy port:

thread = codex.thread_start(
    model=model,
    model_provider="vllmresp",
    config={"model_providers": {"vllmresp": {
        "name": "vllmresp",
        "base_url": f"http://localhost:{port}/v1",
        "wire_api": "responses",
    }}},
    approval_mode=ApprovalMode.auto_review,
    sandbox=Sandbox.full_access,
    ephemeral=True,
    cwd=cwd,
)

Two things worth knowing before you commit to the SDK: it is a prerelease (uv pip install --prerelease=allow openai-codex), and it ships a bundled codex binary (currently 0.137) that will not match a newer system codex. The CLI mode uses the system codex, the SDK mode uses the bundled one — pick one and stay with it.

terminal-bench on a local model

The same wiring runs against terminal-bench when you swap the default CodexAgent for one that injects a custom provider into the container's ~/.codex/config.toml. On Mac + Colima the two knobs are the local LLM URL (host.docker.internal:11434 from inside the container) and the Docker socket path:

export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"
OPENAI_API_KEY=ollama-dummy PYTHONPATH=. tb run \
  -p /tmp/tbcore -t hello-world \
  --agent-import-path tb_codex_local:CodexLocalAgent \
  -m local/gemma4:12b --n-concurrent 1 --no-livestream

Measured with codex-cli + gemma4:12b:

Task	Difficulty	Result
hello-world	easy	PASS (~1 min, 2/2)
heterogeneous-dates	medium	PASS (~10 min, 3/3) — needs --global-agent-timeout-sec 3600

heterogeneous-dates computes the correct answer well before the default 360 s cap but gets cut just before the file write. Raising the cap flips it green. The bottleneck is inference speed on a 12B local, not the model's capability.

Traps worth naming

Symptom	What to try
developer role 400	vLLM without the proxy or the chat_template patch. Check codex points at the proxy on localhost:8731. ollama does not raise this.
unexpected status 200 OK ... ws://	codex 0.142+ falling back to the WebSocket transport. Declare a custom provider with wire_api="responses"; overriding openai_base_url alone is not enough.
model is required (ollama 400)	codex 0.142+ omits model in the body. Pass --model to the proxy so it injects one (run_codex.py already does).
wire_api=chat errors	codex 0.137+ dropped chat and speaks Responses only. Confirm the backend serves /v1/responses (both ollama and vLLM do).
codex refuses to write files	The model is too small. Use a tool-calling model in the gemma4:12b class or above.
openai-codex import fails	Prerelease — uv pip install --prerelease=allow openai-codex. The CLI path works without the SDK.
docker ... Connection aborted, FileNotFoundError	Colima only. export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock".

Closing

Two things carry the whole setup: declare a custom provider with wire_api = "responses", and isolate CODEX_HOME. ollama needs nothing else. vLLM needs one more thing — a home for the developer role — and you get to choose whether it lives in the chat template or in a five-file proxy. The proxy is worth understanding regardless: forwarding the whole /v1/responses schema verbatim, and touching exactly one field, is what keeps this from breaking on the next codex release.

Full code and the terminal-bench adapter are at github.com/ysys143/codex_qwen.