Source: LiteLLM docs: getting started, input params, proxy config.yaml
LiteLLM's docs on the SDK/proxy split, provider-prefixed model names, OpenAI-param translation and drop_params, and the proxy's config.yaml (model_name aliases vs litellm_params.model).
Contents
- What: official documentation pages (“Getting Started”, “Input Params”, “Proxy Config.yaml”)
- Publisher: BerriAI (LiteLLM)
- Date: living docs, read 2026-11-14 (config page describes behaviour as of v1.103.0)
- Original: docs.litellm.ai/docs, completion/input, proxy/configs
Summary
LiteLLM ships two things: a Python SDK that calls 100+ providers through OpenAI-format functions, and a self-hosted proxy that wraps the same code behind an OpenAI-compatible HTTP server. The SDK suits one Python app; the proxy suits a platform team governing LLM access for an organisation. The input-params page explains how OpenAI parameters are translated per provider, and the config page defines how the proxy maps client-facing model names onto provider deployments.
Key claims
- SDK entry points:
completion(),embedding(),image_generation(), plus aRouterfor retries, fallbacks and load balancing. Responses follow the OpenAI Chat Completions shape; streaming (stream=True) yields chunks. - Models are addressed as
<provider>/<model>, e.g.anthropic/...,bedrock/...,vertex_ai/...,ollama/...,azure/<deployment-name>; anopenai/prefix marks any OpenAI-compatible endpoint (the config example points one at a vLLM server). - Provider errors are mapped to OpenAI-style exception classes such as
litellm.RateLimitErrorandlitellm.AuthenticationError, so OpenAI error handling keeps working. - By default an unsupported OpenAI param raises an exception;
litellm.drop_params = True(or per call) silently drops it.additional_drop_paramsandallowed_openai_paramsadjust this per request;get_supported_openai_params()reports support for a model/provider pair. Support varies by model inside a provider. - Any non-OpenAI kwarg is assumed to be provider-specific and passed through into the provider request body.
- In
config.yaml,model_nameis the alias clients send;litellm_params.modelis the provider string. Several entries sharing amodel_nameform a load-balancing group. Top-level sections:model_list,litellm_settings,router_settings,general_settings,credential_list,environment_variables. os.environ/VARcan stand in for any config value and is resolved at startup. Wildcard entries ("*",openai/*) forward arbitrary model names to a provider.general_settings.master_keymakes the proxy require that bearer token. From v1.103.0 a key set inconfig.yamlowns that setting and UI/API writes to it return 400; before that, database values overrode the YAML. Models stored in the database add deployments rather than replacing YAML ones.database_connection_pool_limit(default 10) is per worker, so total Postgres connections scale with instances times workers.