latent.fs / tags / Inference — 3 items

/feed.xml

TAG

#Inference

2026-10 1 post

  1. Every Organization Needs an Agent Runtime

    Part 2 of 4: why even air-gapped organizations will run a general-purpose agent runtime on work PCs, and how to connect it to internal models and tools.

    AI·Agent #37

2026-05 2 posts

  1. Apple Silicon LLM Inference — Five Backends Compared

    Qwen3.5-9B on MLX, llama.cpp, Ollama, omlx, and vLLM Metal. MLX led single requests at 25 tok/s; only omlx batched concurrent requests.

    AI·Infra #13

  2. NVIDIA NIM API — Free Inference for GLM, Kimi, Nemotron, and Gemma 4

    build.nvidia.com serves 100+ open models via a free OpenAI-compatible API, no credit card, up to 40 RPM per model. Setup for Claude Code and Cursor.

    AI·Infra #3

TAGS 190

All 190 tags →