TAG
#vLLM
2026-07 2 posts
-
Driving a Free Colab GPU from the CLI
google-colab-cli drives a free Colab GPU from your laptop. Worked example: serving PaddleOCR-VL on a T4 and running OCR on an invoice over a public URL.
-
Wiring codex to a Local Qwen — ollama, vLLM, and the developer role
Point codex CLI at a local model on ollama or vLLM. ollama needs one config.toml; vLLM rejects the developer role, fixed by a template patch or a proxy.
2026-06 1 post
-
Speeding up LLM inference with MTP and diffusion
Gemma 4 and Qwen 3.6 on one H100 at fp8: MTP ran 1.43–2.62x faster with GPQA accuracy effectively unchanged; diffusion was ~4x faster but less accurate.
2026-05 1 post
-
Apple Silicon LLM Inference — Five Backends Compared
Qwen3.5-9B on MLX, llama.cpp, Ollama, omlx, and vLLM Metal. MLX led single requests at 25 tok/s; only omlx batched concurrent requests.
No matching posts.