跳到正文- Tokenization
- Source: Vercel AI Gateway docs (overview, model fallbacks, budgets)
- Source: RouteLLM, an open-source framework for cost-effective LLM routing (LMSYS blog)
- Source: Redis LangCache docs (overview and API/SDK examples)
- Source: Optimizing for cost and intelligence (Anthropic)
- Source: OpenRouter FAQ (fees, BYOK, free-model limits)
- Source: OpenAI — Flex processing guide
- Source: LiteLLM proxy (AI Gateway) docs: overview, virtual keys, caching
- Source: LiteLLM docs: budgets and rate limits (proxy users, teams, keys)
- Source: Hybrid LLM, cost-efficient and quality-aware query routing (Ding et al., ICLR 2024)
- Source: GPTCache, an open-source semantic cache for LLM applications (Fu & Feng, 2023)
- Source: GPT Semantic Cache (Regmi & Pun, 2024)
- Source: FrugalGPT (Chen, Zaharia, Zou 2023)
- Source: Category-aware semantic caching for heterogeneous LLM workloads (Wang et al., 2025)
- Semantic Caching
- Provisioned Throughput
- Prompt Caching
- Model Selection
- Model Routing and Fallbacks
- LLM Gateways
- LiteLLM
- Cost and Latency Optimization
- Batch Processing