---
ai_assisted: true
title: "Wiring codex to a Local Qwen — ollama, vLLM, and the developer role"
description: "Point codex CLI at a local model on ollama or vLLM. ollama needs one config.toml; vLLM rejects the developer role, fixed by a template patch or a proxy."
pubDate: 2026-07-02
category: "AI"
subcategory: "Agent"
tags: ["codex", "LLM", "vLLM", "ollama", "Qwen", "gemma", "terminal-bench"]
serial: 32
draft: false
published: true
---

OpenAI's `codex` CLI defaults to OpenAI's cloud, but the same binary happily talks to a local ollama or vLLM once two knobs are set right. On ollama the entire setup is a single `config.toml`. On vLLM there is exactly one thing that breaks — the `developer` role — and it can be fixed either at the server (chat template patch) or at the edge (a tiny proxy). This post walks the shortest working paths and shows why the vLLM breakage happens.

---

## The wiring in one picture

`codex` sends `/v1/responses` requests. Any backend that serves that endpoint can host it:

```
[ollama]       codex ─(/v1/responses)──────────────────▶ ollama:11434
[vLLM direct]  codex ─(/v1/responses)──────────────────▶ vLLM:8000      (with chat_template patch)
[vLLM proxy]   codex ─(/v1/responses)─▶ proxy.py ─(fix)─▶ vLLM:8000      (when the server is shared)
```

Two things have to hold on every path:

1. **A custom provider with `wire_api = "responses"`.** codex 0.137+ forces the Responses API, and 0.142+ will fall back to a WebSocket transport on the default provider if you only override `openai_base_url`. Declare the provider explicitly.
2. **An isolated `CODEX_HOME`.** Point codex at the repo's `.codex/` instead of `~/.codex` so personal hooks, trust files, and logs stay out of the experiment.

Everything else is a variation on these two.

## Five-minute quick start with ollama

ollama needs no proxy. It accepts the `developer` role that codex sends and serves `/v1/responses` directly.

**Prep**: `codex --version` (0.137+) and one tool-calling model, e.g. `ollama pull gemma4:12b`.

Put the model name in `.codex/config.toml`:

```toml
model = "gemma4:12b"
model_provider = "local_ollama"

[model_providers.local_ollama]
name = "Local Ollama"
base_url = "http://localhost:11434/v1"
wire_api = "responses"
```

Then run codex with `CODEX_HOME` pointing at that isolated config:

```bash
CODEX_HOME="$PWD/.codex" codex exec \
  --skip-git-repo-check --ephemeral \
  --dangerously-bypass-approvals-and-sandbox \
  "Reply with exactly the single word: PONG"
```

If `PONG` comes back, wiring is good. Swap in a real coding prompt with `-C /tmp/work` and the model starts writing files.

One caveat: model size matters. A `qwen3:0.6b` is fine for smoke-testing the wire, but real coding needs a tool-calling model in the `gemma4:12b` class or above.

## Even shorter — ollama's built-in codex launcher

Recent ollama ships an agent launcher. Running `ollama` with no subcommand opens a menu of coding agents — OpenCode, Hermes, **Codex**, Cline, Qwen Code, and more. Selecting `Launch Codex (gemma4:12b)` spawns codex 0.142.5 already wired to an ollama-served model, no `config.toml` edit required.

![ollama prompt](/images/codex-local-qwen/01-launch.png)

![Launch Codex highlighted in the agent menu](/images/codex-local-qwen/02-menu.png)

![codex trust dialog](/images/codex-local-qwen/03-trust.png)

What the zero-config path costs:

- **User-level config, no isolation.** The launcher does not set `CODEX_HOME`. codex reads `~/.codex/config.toml`, your global hooks, and your personal trust store. If a project-local `.codex/config.toml` is present in the launched cwd, codex quietly drops its provider keys and warns:

  ```
  Ignored unsupported project-local config keys in <cwd>/.codex/config.toml:
    model_provider, model_providers.
  If you want these settings to apply, manually set them in your user-level config.toml.
  ```

  Provider selection is a user-level decision on this path — project-local `.codex/` from the previous section stops being the source of truth. That is a deliberate boundary: a project cannot silently repoint codex at a different backend just by shipping a config file.

  ![ignored project-local config warning and YOLO mode at codex startup](/images/codex-local-qwen/04-warning.png)

- **The rewrite is real, and codex-specific.** In one measured run the launcher replaced `~/.codex/config.toml` with a template pointing at a non-existent tag (`gemma4:12b`) and `wire_api = "responses"` — which does not match ollama's chat-completions shape. The next codex call from an unrelated project hung with no output. The hazard is codex-schema-specific: codex keeps its active provider as a single top-level `model_provider =` key, so a file-level template swap flips the live route. Peer agents whose config separates registration from activation were left alone in the same session — for example Claude Code's `~/.claude.json` uses a `providerProfiles[]` list plus a separate `activeProviderProfileId` pointer, and the launcher added Ollama there as an inactive profile only. Snapshot the good config before running the launcher: `cp ~/.codex/config.toml ~/.codex/config.openai.toml`, restore with `cp` if it swaps.

- **YOLO mode by default.** The session opens with approvals and sandbox off. Fine on a personal laptop, less fine on shared machines.

- **ollama only.** The launcher targets ollama's own runtime. For vLLM the wiring below still applies.

![codex completing a parser.py + pytest task on gemma4:12b via ollama](/images/codex-local-qwen/05-result.png)

Pick the launcher when the use case is one laptop, one ollama model, and no need to keep experiments apart. Keep the isolated `CODEX_HOME` flow when you want per-project sandboxes, side-by-side model comparisons, or the vLLM/proxy path.

## vLLM — two ways past the developer role

Point codex at a vanilla vLLM and the first request dies with `400 "Unexpected message role."` The role in question is `developer`, which codex uses for system-level instructions. Two clean fixes:

**A. Patch the chat template (server-owned vLLM).** If you control the server, add a `developer` branch to the Jinja template and pass it via `--chat-template`. This is the root-cause fix, no proxy on the path.

```jinja
{%- elif message.role == "developer" %}
    {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
```

**B. Front the server with a small proxy (shared vLLM).** When the vLLM instance is shared and you cannot restart it, sit an edge proxy in front and rewrite `developer` items into `instructions` (the leading `system`). The rest of the request is forwarded verbatim.

```bash
# Terminal A — the proxy
python proxy.py --vllm http://<host>:8000/v1 \
  --model Qwen/Qwen3.6-35B-A3B-FP8 --port 8731

# Terminal B — codex against the proxy
CODEX_HOME="$PWD/.codex" codex exec \
  -c model_providers.local_vllm.base_url="http://localhost:8731/v1" \
  -c model_providers.local_vllm.wire_api="responses" \
  -c model_provider="local_vllm" \
  -m Qwen/Qwen3.6-35B-A3B-FP8 \
  --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox \
  "Summarize the layout of this repo"
```

The proxy's job is deliberately small — pass every field of `/v1/responses` straight through, only touch the `developer` items, inject `model` into the body if codex 0.142+ left it blank. Usage counts, tool calls, and SSE streams all come from vLLM unchanged.

The one-shot runner `run_codex.py` wraps both the proxy and the codex call:

```bash
python run_codex.py \
  --vllm http://<host>:8000/v1 \
  --model Qwen/Qwen3.6-35B-A3B-FP8 \
  --cwd . --prompt "Summarize the layout of this repo" --mode cli
```

A healthy run ends with `[turns=N errors=0]` and codex's final answer.

## Why vLLM rejects the developer role

The rejection does not come from vLLM's Python code. It comes from the model's Jinja chat template — the same file that decides how `system`, `user`, `assistant`, and `tool` become tokens:

```jinja
{%- if message.role == "system" %} ...
{%- elif message.role == "user" %} ...
{%- elif message.role == "assistant" %} ...
{%- elif message.role == "tool" %} ...
{%- else %}{{- raise_exception('Unexpected message role.') }}
```

The 400 body ("Unexpected message role.", period included) matches this `raise_exception` string exactly. Because the failure lives in the template, both fixes work at the right layer: **A** teaches the template a new branch; **B** re-labels the item before it ever reaches the template. Existing traffic (system / user / assistant / tool) is byte-identical either way — the patched template only diverges on the `developer` branch, so shared workloads are unaffected.

If you can upgrade to a recent vLLM, the newer `chat_utils` also recognises `developer` natively, which is another server-side path.

## SDK mode (optional)

The Python `openai-codex` SDK gets wired the same way — a custom `model_providers` entry in the `config` dict pointing at the proxy port:

```python
thread = codex.thread_start(
    model=model,
    model_provider="vllmresp",
    config={"model_providers": {"vllmresp": {
        "name": "vllmresp",
        "base_url": f"http://localhost:{port}/v1",
        "wire_api": "responses",
    }}},
    approval_mode=ApprovalMode.auto_review,
    sandbox=Sandbox.full_access,
    ephemeral=True,
    cwd=cwd,
)
```

Two things worth knowing before you commit to the SDK: it is a prerelease (`uv pip install --prerelease=allow openai-codex`), and it ships a bundled codex binary (currently 0.137) that will not match a newer system `codex`. The CLI mode uses the system codex, the SDK mode uses the bundled one — pick one and stay with it.

## terminal-bench on a local model

The same wiring runs against terminal-bench when you swap the default `CodexAgent` for one that injects a custom provider into the container's `~/.codex/config.toml`. On Mac + Colima the two knobs are the local LLM URL (`host.docker.internal:11434` from inside the container) and the Docker socket path:

```bash
export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"
OPENAI_API_KEY=ollama-dummy PYTHONPATH=. tb run \
  -p /tmp/tbcore -t hello-world \
  --agent-import-path tb_codex_local:CodexLocalAgent \
  -m local/gemma4:12b --n-concurrent 1 --no-livestream
```

Measured with `codex-cli + gemma4:12b`:

| Task | Difficulty | Result |
|------|------------|--------|
| hello-world | easy | PASS (~1 min, 2/2) |
| heterogeneous-dates | medium | PASS (~10 min, 3/3) — needs `--global-agent-timeout-sec 3600` |

`heterogeneous-dates` computes the correct answer well before the default 360 s cap but gets cut just before the file write. Raising the cap flips it green. The bottleneck is inference speed on a 12B local, not the model's capability.

## Traps worth naming

| Symptom | What to try |
|---|---|
| `developer` role **400** | vLLM without the proxy or the chat_template patch. Check codex points at the proxy on `localhost:8731`. ollama does not raise this. |
| `unexpected status 200 OK ... ws://` | codex 0.142+ falling back to the WebSocket transport. Declare a custom provider with `wire_api="responses"`; overriding `openai_base_url` alone is not enough. |
| `model is required` (ollama 400) | codex 0.142+ omits `model` in the body. Pass `--model` to the proxy so it injects one (`run_codex.py` already does). |
| `wire_api=chat` errors | codex 0.137+ dropped `chat` and speaks Responses only. Confirm the backend serves `/v1/responses` (both ollama and vLLM do). |
| codex refuses to write files | The model is too small. Use a tool-calling model in the `gemma4:12b` class or above. |
| `openai-codex` import fails | Prerelease — `uv pip install --prerelease=allow openai-codex`. The CLI path works without the SDK. |
| `docker ... Connection aborted, FileNotFoundError` | Colima only. `export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"`. |

## Closing

Two things carry the whole setup: declare a custom provider with `wire_api = "responses"`, and isolate `CODEX_HOME`. ollama needs nothing else. vLLM needs one more thing — a home for the `developer` role — and you get to choose whether it lives in the chat template or in a five-file proxy. The proxy is worth understanding regardless: forwarding the whole `/v1/responses` schema verbatim, and touching exactly one field, is what keeps this from breaking on the next codex release.

Full code and the terminal-bench adapter are at [github.com/ysys143/codex_qwen](https://github.com/ysys143/codex_qwen).