Source: vLLM docs: OpenAI-Compatible Server
Official reference for vLLM's OpenAI-compatible API: auth, extra sampling parameters, structured outputs, generation-config defaults.
Contents
- What: official documentation (latest, read 2026-11-14)
- Author: vLLM project
- Date: 2026-11-14 (accessed)
- Original: vLLM docs: OpenAI-Compatible Server
Summary
How to point an OpenAI client at vllm serve, which request fields go beyond the OpenAI spec, and which server defaults surprise people.
Key claims
- Examples call
http://localhost:8000/v1;--api-keyorVLLM_API_KEYsets a key. - The key only guards
/v1,/v2,/inferenceand/cohereprefixes; other routes such as/invocationsare unauthenticated, so the docs recommend a reverse proxy. - vLLM-only parameters (
top_k,min_p,repetition_penalty, …) go in the OpenAI client’sextra_bodyor straight into the JSON payload. - Structured outputs: a
structured_outputsfield (e.g.{"choice": [...]}) orresponse_formatwithjson_schema. - By default the server applies the model repo’s
generation_config.json, which can override the sampling defaults you expect;--generation-config vllmdisables this. - Request-level chat template controls:
chat_template,chat_template_kwargs,add_generation_prompt,continue_final_message. - A non-zero
priorityrequires priority scheduling on the server.