Prefix caching is the inference server keeping the attention state computed for a request's opening tokens, so a later request with the same byte-identical prefix reuses it instead of recomputing prefill.
Why builders care
Nearly every production deployment repeats the same system prompt, tool definitions, or few-shot prefix on every call, and without reuse the server recomputes that whole prefix each time. Prefix caching turns the repeat into a lookup: matching blocks are reused, fresh compute starts at the first mismatch, and time-to-first-token drops on long stable prefixes. It is the cross-request sibling of the KV cache, and the mechanism providers later bill as prompt caching. For the full four-layer comparison, read the caching layers guide.
Local relevance
On your own hardware the knob is server configuration, not application code. In vLLM, automatic prefix caching ships enabled on the current engine and is controlled with the command-line pair for enabling and explicitly disabling it, with a separate option selecting the block-hash algorithm; in llama.cpp, reusable prompt state is managed with a prompt-cache file plus the option that reuses it across sessions. Keep layouts stable-content-first — frozen instructions and documents up front in a fixed order, per-request values at the end — because any reordering or early timestamp silently busts the match. For the VRAM math that decides what fits alongside the cache, read quantized models explained.