Wiki
An LLM-maintained knowledge base on cutting-edge AI.
A knowledge base on cutting-edge AI, maintained by an LLM from sources I collect. Each concept page cites its sources. See the log for what changed and when.
Concepts
Forward deployed engineering
- AI Governance and Compliance: The regulatory and standards layer for enterprise AI: EU AI Act obligations and dates, NIST AI RMF and its GenAI profile, ISO/IEC 42001, SOC 2 assurance, and bank model risk management, with how they fit together.
- AI Use-Case Discovery: How to find, qualify, scope and prioritise LLM use cases in an organisation: where to look, fit tests for LLMs and agents, impact/effort scoring, success metrics and build-vs-buy.
- Data Retention and Residency: What zero data retention (ZDR) and data residency actually guarantee on Anthropic, OpenAI, Google (Gemini API and Vertex AI), Azure and Bedrock: which stores survive ZDR, which features break it, how regional processing differs from regional storage, and where DPAs and BAAs fit.
- Enterprise LLM Deployment: Running LLMs inside an enterprise’s security perimeter: data retention and ZDR, residency, identity and RBAC, audit logging, PII handling, network isolation, and how to roll out and drive adoption.
- Forward Deployed Engineering: The forward deployed engineer (FDE) role: a product-grade software engineer embedded with one customer to ship production systems, born at Palantir and now central to how OpenAI, Anthropic and AI startups deploy LLMs.
- POC to Production: Taking an LLM proof of concept to a dependable production system: why most pilots stall, the stage gates (evals, guardrails, integration, rollout), human-in-the-loop design, change management and the post-launch data flywheel.
Models
- Batch Processing: Asynchronous vendor batch APIs that run large request sets at about 50% off within a 24-hour window: limits, lifecycle, result handling and caching interplay across Anthropic, OpenAI and Gemini.
- Computer-Use Agents: Agents that operate GUIs and browsers through screenshots, mouse and keyboard (Anthropic computer/browser use, OpenAI CUA, Playwright-based agents): the action loop, grounding, benchmarks, reliability, prompt-injection risk and sandboxing.
- Context window: The maximum number of tokens a model can attend to in one request (input plus output), what counts against it, vendor limits as of 2026-11, overflow behaviour, and why the usable window is smaller than the advertised one.
- Extended Thinking: Claude’s API mechanism for visible-but-summarized reasoning: adaptive thinking with effort (current) or a fixed budget_tokens (legacy), thinking blocks with signatures, and interleaved thinking between tool calls.
- Frontier Model Landscape: The proprietary frontier model families (Anthropic Claude, OpenAI GPT, Google Gemini, xAI Grok) as of November 2026: tiers, context windows, prices, and how vendors differentiate.
- Function Calling: The API protocol by which a model emits structured requests to run developer-defined tools, and how Anthropic, OpenAI and Gemini differ on schemas, tool choice and parallel calls.
- Hallucination: When an LLM produces fluent output that is false or unsupported: its types, why training makes it inevitable, how to detect it, and how to reduce it.
- LLM limitations: The systematic failure modes of LLMs (hallucination, knowledge cutoff, tokenization artifacts, non-determinism, sycophancy, position bias, long-context decay, refusals) with causes and mitigations.
- LLM Non-Determinism: Why the same prompt at temperature 0 can return different completions: batch-size-dependent kernels, floating-point precision, MoE routing, serving changes and seeds; what each vendor guarantees; and how to engineer around it.
- Long-context degradation: Why LLM accuracy falls as inputs grow and depends on where information sits: lost-in-the-middle, needle-in-a-haystack limits, effective context length and context rot.
- Mixture of Experts (MoE): Sparse transformer architecture where a router sends each token to a few of many expert FFNs: compute follows active parameters, memory follows total parameters, and it now underlies almost every large open-weight model.
- Model Selection: How to choose an LLM for a workload: criteria, eval-driven selection, effort tuning before model switching, tier routing and cascades, and cost per completed task.
- Multimodal Models: Models that take images, documents and audio as input (and sometimes produce images or speech): how inputs become tokens, per-vendor limits and costs, image generation, speech models, and failure modes.
- Open-Weight Models: LLMs whose trained weights you can download and run yourself (Qwen, DeepSeek, Kimi, Mistral, Meta Muse/Llama, Gemma, gpt-oss): architectures, sizes, licences and trade-offs as of November 2026.
- Prompt Caching: Reusing the stored KV state of a repeated prompt prefix to cut input cost and latency: mechanics, breakpoints, TTLs and pricing across Anthropic, OpenAI and Gemini.
- Reasoning Models: LLMs trained with reinforcement learning to produce a long hidden chain of thought before answering, trading tokens and latency for accuracy on verifiable, multi-step problems.
- Sampling parameters: How temperature, top-k, top-p, min-p, penalties, stop sequences and max tokens shape what an LLM generates, how each vendor exposes them as of 2026-11 (Anthropic removed temperature/top_p/top_k), and why temperature 0 is still not deterministic.
- Streaming: Incremental delivery of model output over server-sent events: each vendor’s event grammar, how tool calls and thinking stream, mid-stream errors and recovery.
- Structured Outputs: Forcing model output to match a JSON Schema via grammar-constrained decoding, how it differs from JSON mode, and each vendor’s schema subset, limits and failure modes.
- Test-Time Compute: Spending more inference compute per query (longer reasoning, many samples plus selection, search against a verifier) to raise accuracy: a scaling axis separate from model size.
- Tokenization: How LLMs split text into subword tokens (BPE, byte-level BPE, Unigram, WordPiece, SentencePiece), why token counts differ across vendors, models and languages, and how to count them correctly.
- Transformer architecture: How a decoder-only transformer turns tokens into a next-token distribution: embeddings, causal self-attention, feed-forward blocks, positional encoding (RoPE) and the attention variants that make inference affordable.
- Voice Agents: Agents that converse in real-time speech: cascaded STT-LLM-TTS vs speech-to-speech vs full-duplex architectures, the latency budget, turn detection and interruptions, transports, tool calls and evaluation.
Prompting
- Chain-of-thought: Prompting a model to generate intermediate reasoning before its answer, plus its extensions (zero-shot CoT, self-consistency, tree of thoughts) and how built-in thinking changed it.
- Constrained Decoding: How grammar- and JSON-schema-constrained generation works inside the decoder (token masks, FSM and CFG compilation, XGrammar, llguidance, Outlines), its real latency and quality costs, provider strict-mode coverage and failure modes.
- Few-shot prompting: Steering an LLM with a handful of input-output demonstrations in the prompt (in-context learning): how it works, how to pick and format examples, and its biases.
- Instructor and Outlines: The two Python approaches to typed LLM output: Instructor validates Pydantic models client-side and re-asks on failure; Outlines constrains generation token by token (or delegates to provider strict modes). Mechanics, modes, costs, failure modes and when to use which.
- Prompt engineering: Designing, structuring, testing and versioning the instructions and inputs an LLM sees in a single call, per Anthropic, OpenAI and Google guidance.
- System prompts: The privileged, operator-level instructions that frame a conversation: how vendors implement them (system, developer, instructions), their precedence over user turns, what belongs in them, and their limits.
Retrieval and data
- Agentic RAG: RAG where an LLM controls retrieval: deciding whether, what and how often to retrieve, grading results and correcting course; covers routers, corrective RAG (CRAG), Self-RAG, adaptive RAG and just-in-time agentic search.
- ANN Indexes: Approximate nearest-neighbour index structures (HNSW, IVF, PQ and other quantizers, DiskANN) that trade a little recall for orders-of-magnitude faster vector search, and how to tune them.
- BM25: The Okapi BM25 lexical ranking function: IDF times a saturating, length-normalized term frequency, tuned by k1 and b; the default scorer in Lucene-family engines and the baseline every neural retriever is measured against.
- Chunking: Splitting documents into retrievable units for RAG: fixed, recursive, semantic, LLM and late chunking, the size/overlap trade-off, contextual chunk headers, and how to measure chunk quality.
- Contextual Retrieval: Index-time enrichment that prepends document-level context (an LLM-written blurb or a templated chunk header) to every chunk before embedding and BM25 indexing; Anthropic measured 35-67% fewer retrieval failures.
- Document Processing: Document AI for LLM systems: turning PDFs, scans and office files into structured text with layout detection, OCR and table-structure models (Docling, Unstructured, LlamaParse) or with vision-language models, plus how to choose, evaluate and fail safely.
- Embeddings: Fixed-length vectors from an encoder model that place semantically similar inputs close together; the representation behind semantic search and RAG.
- GraphRAG: RAG over an LLM-built knowledge graph with hierarchical community summaries, built for multi-hop and whole-corpus questions that flat top-k retrieval cannot answer; plus cheaper variants like LazyGraphRAG.
- Hybrid search: Combining lexical BM25 and dense vector retrieval and fusing the two rankings, usually with reciprocal rank fusion (RRF) or a weighted score blend.
- Late-Interaction Retrieval: Multi-vector retrieval (ColBERT, ColBERTv2/PLAID, ColPali/ColQwen for page images) that keeps one vector per token or image patch and scores with MaxSim: how it works, what it costs to store, how to scale it, and when it beats single-vector embeddings.
- Query Transformation: Rewriting the user’s question before retrieval (rewriting, expansion, HyDE, multi-query fusion, step-back, decomposition, self-query metadata filters): how each works, what it costs, and when it helps or hurts.
- Reranking: A second retrieval stage that re-scores a wide candidate set with a more expensive model (cross-encoder, late-interaction or LLM) to put the truly relevant chunks on top.
- Retrieval-augmented generation (RAG): Grounding an LLM’s answer in documents fetched at query time: the ingest, retrieve, augment, generate pipeline, its add-ons (query rewriting, HyDE, contextual retrieval), its CAG alternative and how to evaluate it.
- Text-to-SQL: Translating natural-language questions into executable SQL with LLMs: schema linking, value retrieval, candidate generation and self-correction, semantic layers as the governed alternative, the Spider/BIRD/Spider 2.0 benchmarks and why their numbers overstate production accuracy.
- Vector Databases: Systems that store embeddings with metadata and answer filtered nearest-neighbour queries at scale; how pgvector, Pinecone, Weaviate, Qdrant, Milvus and Chroma differ.
Training and fine-tuning
- Fine-tuning: Continuing to train a pre-trained LLM’s weights on task data (SFT, preference or RL) to change how it behaves; when to use it versus prompting or RAG, how to prepare data, and how it fails.
- Knowledge Distillation: Training a small student model to imitate a larger teacher, from Hinton’s soft targets to LLM sequence-level, black-box, reasoning and on-policy distillation.
- LLM Training Pipeline: How a frontier LLM is produced, stage by stage (data curation, pretraining, mid-training and long-context extension, SFT, preference tuning, RLVR, constitutional and spec-based safety training), and how each stage explains behaviour you see in production.
- PEFT and LoRA: Parameter-efficient fine-tuning: freeze the base model and train a small adapter. How LoRA, QLoRA, DoRA and rsLoRA work, their key hyperparameters, and when they match full fine-tuning.
- Preference Optimization: Post-training that pushes a model toward preferred or rewarded outputs: RLHF with PPO, DPO and its variants, GRPO and RL with verifiable rewards, and RLAIF / Constitutional AI.
- Reinforcement Fine-Tuning: Post-training an LLM with reinforcement learning against a programmatic or model-based grader (RLVR, GRPO and its variants, managed RFT APIs), which is how reasoning models are trained and how narrow expert tasks are sharpened.
- Scaling Laws: Empirical power laws linking LLM loss to parameters, training tokens and compute: Kaplan vs Chinchilla, the ~20 tokens/parameter rule, why production models are deliberately over-trained for cheaper inference, and the limits of extrapolation.
- Synthetic Data Generation: Using LLMs to write training and evaluation data (self-instruct, evolution, persona and template prompting, teacher distillation, rejection sampling), plus the filtering, decontamination and licensing work that decides whether it helps or hurts.
Inference and serving
- Hugging Face Ecosystem: The Hugging Face Hub and its libraries (transformers, datasets, PEFT, safetensors, huggingface_hub) plus its hosted inference (Inference Providers, Inference Endpoints): what each does, key APIs, model cards and licences, pitfalls.
- KV Cache: The per-token attention keys and values an LLM stores during generation: how to size it, how engines page and share it (PagedAttention, prefix caching, RadixAttention), and how GQA, MLA and FP8 shrink it.
- LLM Inference Serving: How LLMs are served at scale: prefill vs decode, latency/throughput metrics, continuous batching, PagedAttention, and the vLLM / SGLang / TensorRT-LLM / TGI engines.
- Local LLMs: Running open-weight models on your own laptop, workstation or server with Ollama, llama.cpp, LM Studio or MLX: how the runtimes work, memory math, key settings, failure modes and when local beats an API.
- Quantization: Storing and computing LLM weights, activations and KV cache in fewer bits (FP8, INT8, INT4): how the methods work (GPTQ, AWQ, SmoothQuant, LLM.int8, GGUF), what they cost in accuracy, and which to use where.
- Semantic Caching: Serving a stored LLM response when a new prompt’s embedding is close enough to a cached one: mechanics, threshold tuning, false hits, invalidation, per-user scoping and side channels, how it differs from prompt caching, and realistic hit rates.
- Speculative Decoding: Cutting decode latency by letting a cheap drafter propose several tokens that the target model verifies in one pass, with a rejection rule that keeps outputs identical in distribution; variants include draft models, n-gram, Medusa, EAGLE and MTP.
- vLLM: The open-source LLM serving engine built on PagedAttention: how its V1 engine schedules, caches and parallelizes, what its OpenAI-compatible server does, which flags matter, and how it compares with SGLang and TensorRT-LLM.
Evaluation and observability
- Agent Evaluation: Evaluating multi-step agents rather than single completions: outcome vs trajectory grading, tool-call checks, simulated environments and users, pass@k vs pass^k, and cost and turn budgets.
- Arize Phoenix: Source-available, OpenTelemetry-native tracing and evaluation tool: OpenInference span conventions, phoenix-otel setup, structured-output LLM evaluators, datasets and experiments, free self-hosting, and how it compares with Braintrust, Langfuse and LangSmith.
- Benchmarks: Public LLM benchmarks (MMLU, GPQA, HLE, SWE-bench, τ-bench, ARC-AGI, Arena): what each measures, how it is graded, saturation, contamination, and how to read leaderboards.
- Braintrust: Hosted eval-and-observability platform built around immutable experiments, versioned datasets, reusable scorers and sampled online scoring of production logs; how it works, its architecture, pricing and when to pick it.
- Cost and Latency Optimization: The levers that cut LLM application cost and latency (prompt caching, semantic caching, batching, routing and cascades, smaller models, token budgets, streaming, parallelism) and their trade-offs.
- Error Analysis: The practitioner loop of reading LLM and agent traces, open-coding failures in free text, axial-coding them into a counted failure taxonomy, and turning the top categories into targeted evals.
- Langfuse: Open-source, OpenTelemetry-based LLM engineering platform: tracing, prompt management, datasets, experiments, LLM-as-judge and CI gating, self-hostable on Postgres + ClickHouse.
- LLM Evaluation: Designing evals for LLM apps and agents: golden datasets, code, model and human graders, offline vs online, regression suites, statistics and tooling.
- LLM Observability: Tracing, metrics and feedback for LLM apps and agents: the span data model, OpenTelemetry GenAI conventions, what to log, and how LangSmith, Langfuse and Phoenix differ.
- LLM-as-Judge: Using a language model to grade outputs: pointwise vs pairwise vs reference-guided modes, known biases, calibration against humans, and when not to use it.
- OpenTelemetry GenAI Semantic Conventions: The vendor-neutral OTel vocabulary for LLM and agent telemetry: gen_ai.* spans, attributes, events and metrics, opt-in content capture, spec status (still Development, own repo since 2026), the OpenLLMetry and OpenInference dialects, and which backends ingest it.
- Prompt Management: Treating prompts as versioned, deployable artifacts: registries with immutable versions and movable labels, caching and fallbacks, trace linkage, and the eval-gated promotion loop.
- promptfoo: Open-source (MIT) CLI and library for declarative LLM evals and red teaming: a prompts × providers × tests matrix scored by assertions, with exit codes that gate CI.
- RAGAS and DeepEval: The two most-used open-source LLM metric libraries: how their faithfulness, relevancy and context precision/recall metrics and G-Eval are computed, and why same-named metrics can disagree.
- Retrieval Metrics: How to measure a retriever: hit rate, recall@k, precision@k, MRR, MAP, nDCG and RAGAS context precision/recall, what each one rewards, where implementations disagree, and how to build the labelled set they all need.
Agents
- Agent design patterns: Reusable architectures for arranging one or many agents, and how to choose between them.
- Agent guardrails: Sandboxing, permissions, hooks and input handling that bound what an agent can do: the sandbox vs approval-policy split, the lethal trifecta and Rule of Two, SDK permission pipelines and their gaps, scoped credentials and egress control.
- Agent Handoffs: Transferring control of a conversation from one agent to another via a transfer tool call, as opposed to calling an agent as a tool; mechanics in OpenAI Agents SDK, LangGraph, ADK and Agent Framework, context passing and failure modes.
- Agent Instruction Files: Repository and user Markdown files (AGENTS.md, CLAUDE.md, GEMINI.md, Cursor rules) that coding agents load into every session as standing guidance: discovery order, size caps, scoping, and evidence that less is more.
- Agent loop: The core of every AI agent: a model choosing actions in a loop until the task is done, plus the stop conditions, state rules and limits a production loop needs.
- Agent Memory: How agents retain information beyond one context window: short-term thread state, long-term episodic/semantic/procedural stores, the write and retrieval pipelines (Mem0, Letta/MemGPT, LangGraph store, Claude memory tool), and how memory fails.
- Agent Skills: Folders of instructions, scripts and resources (SKILL.md) that an agent discovers by description and loads on demand through progressive disclosure; an open standard read by Claude, Codex, Gemini CLI and Cursor.
- Code Execution Sandboxes: Isolated environments (provider code-interpreter tools, microVM or gVisor sandbox platforms, OS-level sandboxes) that let agents run model-generated code while bounding what it can touch, reach and consume.
- Coding Agents: LLM agents that read, edit, run and verify code in a real repository through a harness of tools, instructions files and sandboxes (Claude Code, Codex, Cursor, Copilot, Devin).
- Context Compaction: Replacing older conversation or agent history with a model-written summary as the window fills: client vs server-side implementations (Anthropic, OpenAI), tool-result clearing, context resets, failure modes and how to choose.
- Context engineering: Deciding what goes into an agent’s finite context window, and when.
- Deep Research Agents: Agents that spend minutes to tens of minutes planning, searching the web in parallel, reading, and synthesising a cited report; architecture, vendor implementations (OpenAI, Google, Anthropic, Perplexity), evaluation, cost and failure modes.
- Durable Agent Execution: Running long-lived agents so they survive crashes, deploys and multi-day waits: journaling model and tool calls, replaying to resume, idempotent side effects, and the Temporal / LangGraph / Inngest / Vercel Workflow / Restate options.
- Harness Engineering: Designing everything around an LLM that turns it into a working agent (instructions, tools, environment, context management, orchestration, feedback) and re-tuning that harness as models change.
- Human-in-the-Loop: Pausing an agent for human approval, editing or input at risky points: durable interrupts, tool approval gates (approve / edit / reject), escalation triggers, reviewer UX and security, and how the major frameworks implement it.
- Letta: Letta (formerly MemGPT), a platform for stateful agents that manage their own memory: in-context memory blocks or git-backed MemFS files, archival and conversation search, sleep-time/dreaming agents that consolidate memory in the background, and a persistent agent server.
- Mem0: Mem0, a drop-in long-term memory layer for agents: an LLM extracts facts from conversations into vector, SQL and entity stores, and retrieval fuses semantic, keyword and entity signals. Covers the 2025 CRUD pipeline, the 2026 ADD-only redesign, graph memory, the benchmark disputes, and Platform vs OSS.
- Model Context Protocol (MCP): Open standard for exposing tools, resources and prompts to any compatible agent.
- ReAct pattern: ReAct (Yao et al., 2022): an LLM agent interleaves free-text reasoning with tool actions and their observations; the ancestor of today’s tool-calling agent loops.
- Subagents: Delegating focused sub-tasks to agents with fresh context windows.
- Tools: Functions an agent can call to act on the world; their design is interface design for a model: sharp descriptions, lean results, readable errors, a small loaded catalogue and untrusted outputs.
- Verification and evals: Feedback from outside the model, for the agent at runtime and for the builder over time: tests, hooks, critics and citation checks; capability vs regression evals, pass@k vs pass^k, judge biases and online evals.
- Web Search APIs for Agents: How agents get live web access: provider-native search tools, LLM-oriented search APIs (Tavily, Exa, Brave, Perplexity, SerpApi), page extraction (Firecrawl, Jina Reader), citation grounding, and the cost, freshness and security trade-offs.
Frameworks and SDKs
- Agent Hooks and Middleware: Deterministic code the harness runs at fixed points in the agent loop (before/after model calls, around tool calls, at stop) to block, rewrite, inject or observe: Claude Code / Agent SDK hooks, LangChain v1 middleware, OpenAI Agents SDK guardrails and lifecycle hooks, Mastra processors, and how they differ.
- AutoGen and Microsoft Agent Framework: Microsoft’s conversation-first multi-agent framework AutoGen (now archived) and its successor, Microsoft Agent Framework, which merges AutoGen with Semantic Kernel around agents and typed graph workflows.
- Claude Agent SDK: Anthropic’s Python/TypeScript library that embeds Claude Code’s agent loop, built-in tools, permissions, sessions, hooks and subagents in your own process.
- CrewAI: Standalone Python multi-agent framework: role-based Crews (agents, tasks, sequential/hierarchical process) for autonomy, and event-driven Flows for deterministic state and control.
- DSPy: Stanford’s framework for programming rather than prompting LMs: typed signatures, composable modules, adapters that compile them to prompts, and optimizers (BootstrapFewShot, MIPROv2, GEPA, BootstrapFinetune) that tune prompts or weights against a metric.
- Google ADK: Google’s open-source, multi-language Agent Development Kit: LLM agents, workflow agents and (since 2.0) a graph-based workflow runtime, with sessions/state, evaluation, A2A and Google Cloud deployment.
- LangChain: Open-source Python/JS framework that wraps LLM providers, tools and retrievers behind one Runnable interface and, since v1.0 (Oct 2025), centres on create_agent plus middleware running on LangGraph.
- LangGraph: Low-level, Pregel-style orchestration runtime for stateful LLM agents: typed state with reducers, nodes and edges, checkpointed super-steps, durable interrupts for human-in-the-loop, time travel and subgraphs.
- LangSmith: LangChain’s hosted (or self-hosted Enterprise) platform for tracing LLM apps as run trees, evaluating them on versioned datasets with code, LLM-judge and human evaluators, and deploying LangGraph agents.
- LlamaIndex: A Python/TypeScript data framework for LLM apps: Documents and Nodes, an ingestion pipeline, index types, retrievers, postprocessors and response synthesizers composed into query engines, plus event-driven Workflows for agents.
- Mastra: Open-source TypeScript framework for AI agents and applications: agents with a provider/model router, typed suspendable workflows, layered memory, processors as guardrails, scorers for evals, and a local Studio, deployable into existing Node servers.
- Microsoft Agent Framework: Microsoft’s open-source .NET/Python (Go preview) SDK that succeeds Semantic Kernel and AutoGen: one AIAgent abstraction over any chat client, a layered middleware/context-provider pipeline, typed graph workflows, and Azure Foundry integration.
- OpenAI Agents SDK: OpenAI’s lightweight, provider-agnostic Python/TS agent framework built on a Runner loop with agents, handoffs, agents-as-tools, guardrails, sessions and built-in tracing.
- Pydantic AI: A typed Python agent framework from the Pydantic team: Agent[Deps, Output], dependency injection via RunContext, validated structured output with retry, usage limits, capabilities, durable execution engines, and test models.
- smolagents: Hugging Face’s minimal Python agent library built around code actions: a CodeAgent writes Python that an AST-walking interpreter or a remote sandbox runs, with a ToolCallingAgent for JSON tool calls, any model backend, MCP and Hub tools, and hierarchical managed agents.
- Vercel AI SDK: Vercel’s TypeScript toolkit for LLM apps: a provider-agnostic Core (generateText/streamText, Output schemas, tools, stopWhen/prepareStep, ToolLoopAgent, WorkflowAgent), UI hooks like useChat over an SSE stream protocol, and harness adapters.
Protocols
- Agent Authentication and Authorization: How agents act for users without holding their keys: delegated vs agent identities, OAuth 2.1 in MCP (discovery, CIMD/DCR, audience-bound scoped tokens, step-up), token exchange and ID-JAG for enterprise SSO, per-user credential brokers (Arcade, Composio, Vercel Connect, Auth0, AgentCore), least privilege and audit.
- Agent2Agent (A2A) Protocol: Open protocol for agents from different vendors to discover each other via Agent Cards and delegate stateful, long-running Tasks over JSON-RPC, gRPC or REST; the horizontal complement to MCP.
- MCP in Practice: How the Model Context Protocol actually works on the wire and in production: primitives, the stateless 2026-07-28 revision, stdio vs Streamable HTTP, OAuth authorization, registries, SDKs and security failure modes.
Security and safety
- Content moderation: Classifying user and model text or images against a harm taxonomy (hate, sexual, violence, self-harm…) to block, flag or route it: taxonomies, scores and severities, the main classifiers, and how to tune them.
- Guardrail tooling: The libraries, classifiers and managed services that check LLM inputs, outputs and tool calls at runtime: NeMo Guardrails, Guardrails AI, Llama Guard / Prompt Guard, Bedrock Guardrails, Azure Content Safety, OpenAI moderation.
- Jailbreaks: Inputs that get a model to produce content its safety training should refuse: why safety training fails, the main attack families (role-play, encodings, adversarial suffixes, many-shot, multi-turn, automated and Best-of-N), defences such as constitutional classifiers, and the over-refusal trade-off.
- Lethal trifecta: Willison’s rule that an agent with private data, untrusted content and an outbound channel can be made to leak that data, Meta’s Agents Rule of Two that generalises it, and how to apply both when designing agents.
- LLM Red Teaming: Systematic adversarial testing of LLM applications, manual and automated (garak, PyRIT, promptfoo), scoped and reported against OWASP LLM/Agentic Top 10 and MITRE ATLAS.
- LLM security risks: The threat model for LLM applications and agents, organised around the OWASP Top 10 for LLM Applications (2026 edition, with the 2025 mapping): what each risk is, how it happens, and the controls that work.
- OWASP Top 10 for LLM and Agentic Applications: OWASP’s two community risk lists for generative AI: the LLM Top 10 (2023-24, 2025 and 2026 editions) and the Agentic Top 10 (ASI01-ASI10), what each entry means, how the editions differ, and how to use them in a design or code review.
- PII and Data Privacy: Engineering personal-data protection in LLM apps: where PII flows (prompts, logs, embeddings, caches), detection and redaction with Presidio, zero data retention, training opt-outs, data residency, and GDPR implications.
- Prompt injection: Attacks that smuggle instructions into an LLM’s context through user input or retrieved content; why it is unsolved at the model level, how exfiltration works, and the defences that actually bound it (spotlighting, Dual LLM, CaMeL, design patterns, Rule of Two, least privilege).
Platforms
- Amazon Bedrock: AWS’s managed, multi-provider foundation-model service: inference APIs, cross-Region profiles, quotas and capacity tiers, Knowledge Bases, Guardrails, AgentCore, and the data-protection model.
- Anthropic Messages API: Claude’s stateless generation endpoint: content-block request/response shape, client vs server tools and pause_turn, adaptive thinking with signed blocks, opt-in caching (breakpoints or automatic), Files API, versioning, limits and porting gotchas.
- Azure AI Foundry: Microsoft’s AI platform (renamed Microsoft Foundry), which includes Azure OpenAI: the Foundry resource model, deployment types and data zones, PTUs and spillover, quota tiers, guardrails, Agent Service and network isolation.
- Gemini API: Google’s Gemini developer API: the stateful step-based Interactions API vs legacy generateContent, built-in tools (Search grounding, code execution, File Search), thinking levels and thought signatures, implicit vs explicit caching, Files API, tiers and the OpenAI-compatible layer.
- Google Vertex AI: Google Cloud’s AI platform, folded into Gemini Enterprise Agent Platform in April 2026: Gemini and Model Garden, endpoint types and residency, PayGo tiers and Provisioned Throughput, RAG Engine, Agent Runtime, and VPC-SC.
- LiteLLM: BerriAI’s open-source Python SDK and self-hosted OpenAI-compatible proxy for 100+ model providers: provider-prefixed model names, a Router with fallbacks and cooldowns, Postgres-backed virtual keys and budgets, callback-based logging, and a 2026 record of supply-chain and proxy CVEs.
- LLM Gateways: A proxy layer between applications and model providers that gives one (usually OpenAI-compatible) API plus centralised keys, budgets, retries, fallbacks, caching and logging; LiteLLM, OpenRouter, Portkey, Vercel AI Gateway, Cloudflare, Kong and Envoy compared.
- Model Routing and Fallbacks: Choosing which model or provider serves each request: reliability fallbacks and 429 handling, plus cost-quality routing (cascades, learned routers like RouteLLM, difficulty-based routing) with measured savings and failure modes.
- OpenAI Responses API: OpenAI’s primary generation endpoint: typed input/output items, optional server-side state, hosted tools, encrypted reasoning, compaction and background mode, and how it differs from Chat Completions.
- Provisioned Throughput: Reserved LLM inference capacity bought in units (Azure PTUs, Vertex GSUs, Bedrock model units and Reserved tier, OpenAI Scale/Reserved Tier) for a fixed term: how each unit converts tokens through burndown weights, how to size it from peak traffic, quota versus capacity, spillover, and when it beats pay-as-you-go.
- Rate Limits and Retries: How LLM APIs meter traffic (RPM, TPM, ITPM/OTPM, daily and spend limits, ramp limits), how to classify 429/503/529 failures, and how to build clients that survive them: backoff with jitter, retry budgets, timeouts for long generations, concurrency caps, queueing and service tiers.
Sources
- Source: 2 OLMo 2 Furious (Team OLMo, Ai2, 2025): Ai2’s fully open OLMo 2 (7B/13B/32B) report, documenting two-stage pretraining with a mid-training anneal on Dolmino Mix 1124, model souping and Tulu 3-style post-training.
- Source: 21 · Chain-of-Verification experiment (Notebook): The owner’s runnable LangChain experiment implementing the four-call CoVe loop against Claude.
- Source: A Careful Examination of LLM Performance on Grade School Arithmetic (Zhang et al., 2024): GSM1k paper: a fresh GSM8k-style test set reveals up to 8-point drops and memorisation-linked overfitting in some model families.
- Source: A Field Guide to Rapidly Improving AI Products (Hamel Husain): Hamel Husain’s 2025 essay arguing that bottom-up error analysis, custom data viewers and binary judgments drive AI product improvement.
- Source: A Holistic Approach to Undesired Content Detection in the Real World: OpenAI’s AAAI-23 paper on building a production moderation classifier: taxonomy, labelling quality, active learning for rare events, robustness.
- Source: A postmortem of three recent issues (Anthropic, 2025): Anthropic’s postmortem of three August-September 2025 infrastructure bugs that degraded Claude output, including a bf16/fp32 precision mismatch and an XLA:TPU approximate top-k miscompilation.
- Source: A Practical Guide to Building Agents (OpenAI): OpenAI’s 2025 guide on when to build an agent, agent components, orchestration patterns, guardrails and human-intervention triggers.
- Source: A sandbox without a network boundary is only half a sandbox (Vercel, 2026): Vercel engineering post arguing egress control and credential brokering are part of the sandbox boundary, not add-ons, and describing its host-side firewall design.
- Source: A small number of samples can poison LLMs of any size (Anthropic): Anthropic, UK AISI and Turing Institute (Oct 2025): about 250 poisoned documents backdoored models from 600M to 13B parameters regardless of data size.
- Source: A Survey on Hallucination in Large Language Models (Huang et al.): ACM TOIS survey giving the standard factuality/faithfulness taxonomy of LLM hallucination, its causes, detection and mitigation.
- Source: A2A and MCP (A2A project docs): A2A documentation page positioning A2A (agent-to-agent, stateful collaboration) as complementary to MCP (agent-to-tool), with an auto-repair-shop example.
- Source: A2A in ADK (ADK docs): ADK’s support for the Agent2Agent protocol: exposing agents, consuming remote agents with RemoteA2aAgent, and when to prefer local sub-agents.
- Source: A2A Protocol Specification v1.0: The Agent2Agent protocol specification: Agent Cards, Tasks, Messages, Parts, Artifacts, nine task states, three protocol bindings, streaming, push notifications and security.
- Source: a2aproject/A2A GitHub repository: The A2A project repository: governance under the Linux Foundation’s Agentic AI Foundation, Apache 2.0 licence and the official SDK list.
- Source: AAIF: A2A v1.0 Builder’s Guide, Part 1: Practitioner guide to A2A v1.0 discovery, task lifecycle, progress tracking, SDK structure, 0.3-to-1.0 breaking changes and common gotchas.
- Source: Adding Error Bars to Evals (Miller, 2024): Statistics paper arguing evals are experiments: report standard errors, cluster them, use paired differences and power analysis.
- Source: ADK agent transfer (adk-python source and LlmAgent docs): How Google ADK implements transfer_to_agent: an enum-constrained transfer tool, target selection from parent/peers/sub-agents, and the injected transfer instructions.
- Source: ADK documentation: Why evaluate agents: Google Agent Development Kit docs on test files, evalsets, trajectory and response metrics, rubric judges, user simulation and the adk eval CLI.
- Source: ADK workflows, graph workflows and LoopAgent (ADK docs): ADK documentation on workflow types: template workflow agents (Sequential, Parallel, Loop), graph-based and dynamic workflows.
- Source: Advancing semantic caching with domain-specific embeddings and synthetic data (Gill et al., 2025): Redis/Virginia Tech paper showing a small ModernBERT model fine-tuned for duplicate-query detection (LangCache-Embed) matches or beats large general embedding models for cache-hit precision.
- Source: Agencies Overhaul Model Risk Management Guidance for Banks (Orrick): Law-firm analysis of the 2026 interagency MRM guidance: narrower model definition, materiality-based tiering, a lighter validation description, non-enforceability, and a carve-out for generative and agentic AI.
- Source: Agent concepts and Agent Pipeline Architecture (Microsoft Agent Framework docs): How an Agent Framework agent run flows through agent middleware, context/history providers, the function-invocation loop, chat middleware and the raw chat client, in .NET, Python and Go.
- Source: Agent Development Kit (ADK) documentation home: Landing page of Google’s open-source Agent Development Kit: languages, features, model support, deployment targets and A2A.
- Source: Agent Development Kits (Loic’s Notebook): The owner’s tutorial comparing Google ADK, AWS Strands Agents, Bedrock AgentCore and LangGraph.
- Source: Agent SDK overview (Claude Code docs): Anthropic’s overview of the Claude Agent SDK: Claude Code’s agent loop, tools and context management as a Python/TypeScript library.
- Source: Agent Skills (Cursor docs): Cursor’s skills support: .cursor/.agents skill folders with Claude/Codex compatibility paths, paths-scoped surfacing, manual-only skills, and migration of rules and commands to skills.
- Source: Agent Skills (Gemini CLI docs): Gemini CLI’s implementation of Agent Skills: four discovery tiers, model-called activate_skill with a user consent prompt, and /skills and gemini skills management commands.
- Source: Agent Skills overview (Claude Platform docs): Anthropic’s reference for how skills load across the API, Claude Code and claude.ai: three-level token costs, VM/filesystem architecture, surface differences, security and data retention.
- Source: Agent Skills specification (agentskills.io): The open format spec for Agent Skills: SKILL.md frontmatter fields and constraints, optional scripts/references/assets folders, progressive-disclosure budgets and a validator.
- Source: AgentDojo: Debenedetti et al. (2024): a dynamic benchmark of 97 agent tasks and 629 prompt-injection test cases in simulated email, banking and travel environments.
- Source: agentevals (LangChain): Open-source library of agent trajectory evaluators: reference trajectory matching in four modes, tool-argument matching rules, and LLM-as-judge trajectory graders.
- Source: Agents (CrewAI docs): CrewAI Agent attributes and defaults (role, goal, backstory, max_iter, delegation, context window handling) and standalone kickoff.
- Source: Agents (OpenAI Agents SDK docs): Agent configuration in the OpenAI Agents SDK: instructions, tools, output types, context, tool-use behaviour, hooks and multi-agent patterns.
- Source: Agents Rule of Two (Meta): Meta (Oct 2025): within one session an agent should hold at most two of untrusted input, sensitive data/systems, and state change or external communication.
- Source: AGENTS.md (open format site): The AGENTS.md site: a plain-Markdown ‘README for agents’ adopted by 60k+ repos and many coding agents, now stewarded by the Agentic AI Foundation under the Linux Foundation.
- Source: AI Agent Authentication and Authorization (draft-klrc-aiagent-auth): IETF draft (July 2026) profiling WIMSE/SPIFFE, OAuth, token exchange, transaction tokens and Shared Signals into best practice for agent identity; no new protocol.
- Source: AI Agents That Matter (Kapoor et al., 2024): Princeton paper arguing agent benchmarks must report cost alongside accuracy, use proper holdouts, and be reproducible; simple baselines rival complex agents.
- Source: AI Evals: Everything You Need to Know (Husain and Shankar): Hamel Husain and Shreya Shankar’s long-form FAQ on evals: error analysis via open and axial coding, agent and multi-step evaluation, synthetic data, CI vs production.
- Source: AI in the Enterprise (OpenAI): OpenAI’s 2025 guide distilling seven enterprise adoption lessons from Morgan Stanley, Indeed, Klarna, Lowe’s, BBVA, Mercado Libre and OpenAI itself, starting with evals.
- Source: AI SDK 7 is now available (Vercel changelog): Vercel’s announcement of AI SDK 7 (2026-06-25): tool approvals, durable WorkflowAgent, timeouts, sandboxes, harness adapters, reasoning control, and Node 22/ESM-only requirements.
- Source: AI SDK documentation (Vercel): Official AI SDK docs: Core (generateText/streamText, Output, tools, stopWhen/prepareStep), ToolLoopAgent, middleware, the provider specification, and AI SDK UI’s useChat and stream protocol.
- Source: Amazon Bedrock AgentCore Identity docs: AWS’s agent identity service: a directory of agent workload identities, a KMS-encrypted token vault, OAuth 2LO/3LO credential providers, and SDK decorators that inject credentials.
- Source: Amazon Bedrock AgentCore — How Runtime Works: Internals of AgentCore Runtime: versions and endpoints, per-session microVMs, 8-hour sessions, V2 snapshot cold starts and inbound/outbound auth.
- Source: Amazon Bedrock AgentCore — Overview: AWS’s modular, framework-agnostic agent platform: Runtime, Harness, Memory, Gateway, Identity, tools, Observability, Evaluations, Policy and more.
- Source: Amazon Bedrock data retention: Bedrock’s data-retention modes (none, default, aws_review, legacy provider_data_share), their resolution order, per-model allowed modes, and SCP enforcement of zero retention.
- Source: Amazon Bedrock — Agents (Classic): The original configuration-driven Bedrock Agents, now closed to new sign-ups and in maintenance mode in favour of AgentCore.
- Source: Amazon Bedrock — Cross-Region Inference: How Bedrock inference profiles route requests across Regions, and the Geographic vs Global trade-off on residency, cost and SCPs.
- Source: Amazon Bedrock — Data Protection and VPC/PrivateLink: Bedrock’s data-handling model (provider-isolated model deployment accounts) and its VPC/PrivateLink guidance.
- Source: Amazon Bedrock — Guardrails: Bedrock’s configurable safety layer: content filters, denied topics, word and PII filters, contextual grounding and Automated Reasoning checks.
- Source: Amazon Bedrock — How Tokens Are Counted Against Quotas: How Bedrock reserves
max_tokensup front and applies output-token burndown multipliers to TPM/TPD quotas. - Source: Amazon Bedrock — Knowledge Bases: Bedrock’s managed RAG service: Managed vs customer-managed knowledge bases, connectors, agentic retrieval and the supported vector stores.
- Source: Amazon Bedrock — Model Customization: AWS documentation for supervised fine-tuning, reinforcement fine-tuning and distillation on Bedrock.
- Source: Amazon Bedrock — Overview (User Guide): AWS’s landing page for Bedrock: the managed multi-provider model service, its five inference API styles and the 2026 model roster.
- Source: Amazon Bedrock — Provisioned Throughput: Bedrock’s reserved-capacity option: hourly-billed model units with no-commitment, 1-month or 6-month terms. Custom models need it unless their base model supports on-demand deployment.
- Source: Amazon Bedrock — Service Tiers: Bedrock’s four request tiers (Reserved, Priority, Standard, Flex) selected per request via
service_tier. - Source: An Empirical Study of Catastrophic Forgetting in LLMs During Continual Fine-tuning: A 2023 study measuring how continual instruction tuning erodes LLMs’ prior knowledge and skills.
- Source: Announcing Mastra 1.0 (Mastra blog): Mastra 1.0 stable release: server adapters, composite storage, processor overhaul, AI SDK v6 support, unified observability, breaking changes and adoption numbers.
- Source: Anthropic API rate limits docs: Anthropic’s rate-limit reference: token-bucket RPM/ITPM/OTPM limits per tier, cache-aware ITPM, ramp limits, spend caps, and the 429 / retry-after / anthropic-ratelimit-* header contract.
- Source: Anthropic Browser Use Tool Docs: Claude API documentation for the browser use toolset: 31 member tools mixing accessibility-tree refs and coordinates, tab state reporting, and a hardening checklist.
- Source: Anthropic Commercial Terms of Service: Anthropic’s API terms: customers own Outputs but may not use the Services to build competing products, including training competing AI models.
- Source: Anthropic Computer Use Tool Docs: Claude API documentation for the computer use toolset: versions, 17 member actions, coordinate scaling, the batch agent loop, screenshot token costs and security guidance.
- Source: Anthropic cookbook: Contextual Retrieval guide: Runnable Anthropic cookbook that reproduces contextual retrieval on 9 codebases, with the contextualizer prompt, prompt-caching cost stats and Pass@k numbers per stage.
- Source: Anthropic Message Batches Docs: Claude API documentation on the Message Batches API: 50% discount, limits, lifecycle, result types and interplay with prompt caching.
- Source: Anthropic PDF Support Docs: Claude API documentation on PDF input: each page sent as extracted text plus a page image, page and size limits, token cost per page and best practices.
- Source: Anthropic Prompt Caching Docs: Claude API documentation on prompt caching: breakpoints, TTLs, pricing multipliers, lookback, invalidation and usage fields.
- Source: Anthropic Streaming Messages Docs: Claude API documentation on SSE streaming: event sequence, delta types, partial JSON tool inputs, errors mid-stream and recovery.
- Source: Anthropic Structured Outputs Docs: Claude API documentation on JSON outputs and strict tool use via grammar-constrained decoding: schema subset, complexity limits, failure modes.
- Source: Anthropic Tool Use Docs: Claude API documentation on tool use: client vs server tools, tool definitions, tool_choice, parallel tool calls and token overhead.
- Source: Anthropic Vision Docs: Claude API documentation on image input: formats, per-request limits, the 28x28-patch visual-token cost model, resolution tiers and stated limitations.
- Source: Anthropic — Claude API errors: Anthropic’s error reference: HTTP codes (429 rate_limit_error, 500, 504 timeout_error, 529 overloaded_error), SDK auto-retry defaults, request IDs, and the long-request guidance (stream or batch beyond ~10 minutes).
- Source: Anthropic — Service tiers (Priority Tier): Anthropic’s legacy service-tier page: Standard, Batch and Priority Tier; Priority commitments in ITPM/OTPM with cache-weighted burndown, 99.5% uptime target,
service_tierauto/standard_only. No longer sold as of late 2026. - Source: Anthropic, Introducing Computer Use (Claude 3.5 Sonnet): Anthropic’s October 2024 announcement of computer use in public beta, with early OSWorld results and stated limitations.
- Source: Anthropic, Updates to Consumer Terms and Privacy Policy: August 2025 change letting Claude Free/Pro/Max users choose whether chats train models, with 5-year retention if they opt in; commercial and API products unaffected.
- Source: Anthropic: Introducing Contextual Retrieval: Anthropic engineering post showing that LLM-written chunk context plus BM25 plus reranking cuts retrieval failures by 67%.
- Source: Apache Lucene BM25Similarity (javadoc and source): Lucene’s default BM25 implementation: k1 = 1.2, b = 0.75, an IDF with +1 inside the log, no (k1+1) factor, and a 256-entry length cache.
- Source: API and data retention (Anthropic): Anthropic’s reference for zero data retention (ZDR), HIPAA readiness, Covered Models that require 30-day retention, and per-feature eligibility on the Claude API.
- Source: API versions (Claude docs): Anthropic’s anthropic-version header policy and the changes in 2026-10-21 (sampling parameters removed, stricter citation and document placement).
- Source: ARC-AGI series and ARC-AGI-3 (ARC Prize Foundation): ARC Prize pages describing the ARC-AGI series and ARC-AGI-3’s move to interactive game environments scored on human action efficiency.
- Source: ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems (Chollet et al., 2025): Paper introducing ARC-AGI-2, grid-transformation puzzles calibrated so humans solve them and frontier models largely do not.
- Source: Arcade authorization docs: Arcade’s docs on authorized tool calling: per-user OAuth via tools.authorize, server-level vs tool-level auth for MCP servers, and user verification against auth-link phishing.
- Source: Are Emergent Abilities of Large Language Models a Mirage? (Schaeffer et al., 2023): NeurIPS 2023 paper arguing that sudden ’emergent’ jumps with scale often come from discontinuous evaluation metrics rather than from the models.
- Source: Are We Done with MMLU? (Gema et al., 2024): Paper estimating 6.49% of MMLU questions are wrong and releasing the re-annotated MMLU-Redux subset.
- Source: Arize Phoenix and OpenInference documentation: Phoenix, an OTel-native open-source tracing and evaluation tool, and OpenInference, the span-kind and attribute conventions its instrumentors emit.
- Source: Arize Phoenix docs: evals, experiments, phoenix-otel, self-hosting, and Arize pricing: Phoenix’s evaluator model (ClassificationEvaluator, Score, pre-built metrics), dataset experiments, register() tracing setup, free ELv2 self-hosting, and Arize AX plan prices.
- Source: Asai et al. 2023: Self-RAG: Paper training an LLM to emit reflection tokens that decide when to retrieve and critique relevance and support.
- Source: Attention Is All You Need (Vaswani et al., 2017): The paper that introduced the Transformer: attention-only sequence modelling with multi-head scaled dot-product attention.
- Source: Auth0 Token Vault docs: Auth0 for AI Agents’ Token Vault: stores users’ third-party OAuth tokens on connected accounts and releases them through refresh-token, access-token or privileged-worker exchanges.
- Source: AutoGen to Microsoft Agent Framework Migration Guide (Microsoft Learn): Side-by-side mapping of AutoGen AgentChat/Core concepts to Microsoft Agent Framework agents and typed data-flow workflows.
- Source: AutoGen v0.4: Reimagining the foundation of agentic AI (Microsoft Research): Microsoft Research post on the AutoGen v0.4 redesign: async event-driven core, layered APIs, observability and AutoGen Studio.
- Source: AutoGen vs CrewAI (Loic’s Notebook): The owner’s tutorial contrasting AutoGen’s conversation-first model with CrewAI’s role-and-task model.
- Source: AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation (Wu et al., 2023): The original AutoGen paper introducing conversable agents and conversation programming for multi-agent LLM apps.
- Source: AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration: MLSys 2024 best paper: protect salient weight channels, identified from activations, via equivalent scaling.
- Source: AWS HIPAA Eligible Services Reference: AWS’s list of services that may process PHI under the AWS BAA; Amazon Bedrock and Bedrock AgentCore are listed with no feature exclusions.
- Source: AWS Services Field Guide (Notebook): The owner’s field guide to core AWS services, with a Bedrock entry: what it solves, its limits, and SageMaker vs Bedrock.
- Source: Axolotl Documentation: Documentation for Axolotl, a YAML-configured open-source framework for LLM fine-tuning and post-training.
- Source: Azure AI Content Safety documentation: Microsoft Learn docs for Azure AI Content Safety: harm categories with 0-7 severity, Prompt Shields for user and document attacks, groundedness, protected material, limits.
- Source: Azure API Management llm-semantic-cache-lookup policy reference: Microsoft’s gateway policy for semantic response caching: a distance-style score threshold (start near 0.05), vary-by partitioning, system-message stripping, max-message-count, and an explicit warning about wrong or unsafe hits.
- Source: Background mode (OpenAI guide): OpenAI’s guide to running long Responses API calls asynchronously with background: true, polling, cancellation and resumable streaming.
- Source: BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of IR Models (Thakur et al.): The 18-dataset zero-shot retrieval benchmark that standardised nDCG@10 and showed annotation pools biased toward lexical retrievers under-score dense models.
- Source: Benchmarking AI Agent Memory: Is a Filesystem All You Need? (Letta, 2025): Letta shows a GPT-4o-mini agent with only file-search tools scoring 74.0% on LoCoMo, above Mem0’s best reported score, and argues LoCoMo says little about memory.
- Source: Best practices for Claude Code (Anthropic docs): Anthropic’s official guidance for Claude Code: tell it what it can’t see, give it a check to run, enforce hard rules with hooks and deny rules, and bound unattended runs.
- Source: Best-of-N Jailbreaking: Hughes et al. (2024): resample randomly augmented prompts (shuffled, recapitalised) until one works; 89% ASR on GPT-4o and 78% on Claude 3.5 Sonnet at N=10,000.
- Source: Beyond Chinchilla-Optimal: Accounting for Inference in Scaling Laws (Sardana et al., 2024): ICML 2024 paper (MosaicML/Databricks) extending Chinchilla to include inference cost, showing high-demand models should be smaller and trained on far more tokens.
- Source: BFCL V3: Multi-Turn and Multi-Step Function Calling (Berkeley): Berkeley Function Calling Leaderboard post explaining why exact call matching fails for multi-turn agents and introducing state-based plus subset-matched response checks.
- Source: Binary and Scalar Embedding Quantization (Hugging Face blog): Hugging Face / mixedbread post measuring binary and int8 quantization of embeddings: 32x and 4x memory savings, speed-ups, and recall retained with rescoring.
- Source: BIRD Benchmark (Li et al., NeurIPS 2023) and Leaderboard: Text-to-SQL benchmark over 95 large, dirty real-world databases with external-knowledge evidence and an efficiency metric; the 2026 leaderboard tops out near 83% execution accuracy against 93% for humans.
- Source: Braintrust autoevals (GitHub): MIT-licensed scorer library behind Braintrust: LLM-classifier, RAG, embedding and heuristic scorers usable without the platform.
- Source: Braintrust docs: Eval SDK quickstart: Braintrust’s Eval() abstraction of data, task and scores, producing comparable experiments.
- Source: Braintrust docs: Prompts: Braintrust prompt loading by slug, version and environment, build() without calling the model, version recording on spans, and cache fallback semantics.
- Source: Braintrust platform docs: experiments, scorers, datasets, online scoring, architecture, pricing: Braintrust’s documentation on comparing experiments, scorer types, versioned datasets, online scoring rules, the control/data-plane split with Brainstore, deployment options, Loop, and plan pricing.
- Source: Brave Search API product page: Brave’s API offering on its own independent web index: endpoints including LLM Context for agent grounding, per-1,000-request pricing, rate limits and storage rights.
- Source: Bruch, Gai & Ingber 2022: An Analysis of Fusion Functions for Hybrid Retrieval: Paper finding that a tuned convex combination of lexical and semantic scores beats RRF, and that RRF is parameter-sensitive.
- Source: Build LLM Apps tutorial (Notebook): The owner’s tutorial on LangChain, LangGraph and LangSmith, including zero-code LangSmith tracing and a sample run tree with per-step latency, tokens and cost.
- Source: Build skills (OpenAI Codex docs): How Codex implements the Agent Skills standard: .agents/skills locations, $skill invocation, a 2%-of-context listing budget, openai.yaml policy, and plugins for distribution.
- Source: Building agents with the Claude Agent SDK (Anthropic): Anthropic’s engineering post announcing the rename of the Claude Code SDK to the Claude Agent SDK and describing its gather-act-verify loop.
- Source: Building durable AI agents (Vercel knowledge base guide): Vercel guide (Oct 2026) on running an AI SDK WorkflowAgent on the Workflow SDK: step boundaries, approval hooks with deadlines, idempotency keys from step IDs, and deployment pinning.
- Source: Building effective agents (Anthropic): Anthropic’s guide to agentic systems: prefer the simplest solution, distinguish workflows from agents, and start with raw LLM APIs before adopting a framework.
- Source: C-Pack: Packed Resources for General Chinese Embeddings (BGE): BAAI paper releasing the C-MTEB benchmark, the C-MTP training set and the BGE embedding model family (via FlagEmbedding).
- Source: Calibrate Before Use: Improving Few-Shot Performance of Language Models: Zhao et al. (2021): few-shot accuracy swings with example choice, order and format because of recency and common-token biases; contextual calibration corrects it.
- Source: Category-aware semantic caching for heterogeneous LLM workloads (Wang et al., 2025): Argues that thresholds, TTLs and quotas should vary by query category because repetition, embedding density and staleness differ; gives a break-even model for remote vs local vector search.
- Source: Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: Wei et al. (2022): few-shot exemplars with intermediate reasoning steps sharply improve large models on arithmetic, commonsense and symbolic reasoning.
- Source: Chain-of-Verification Reduces Hallucination in Large Language Models (Meta, 2023): Meta AI paper introducing CoVe: draft, plan verification questions, answer them independently, then revise.
- Source: Chan et al. 2024: Don’t Do RAG, Cache-Augmented Generation: WWW 2025 short paper proposing CAG: preload the whole knowledge base and precompute its KV cache instead of retrieving.
- Source: Chat Completions API reference (OpenAI): OpenAI’s reference for chat completion parameters: temperature, top_p, stop, max_completion_tokens, seed, penalties, logit_bias and logprobs.
- Source: Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (Chiang et al., 2024): Paper describing Chatbot Arena’s crowdsourced, anonymous pairwise battles and statistical ranking of LLMs.
- Source: Chen et al. 2024: BGE-M3 (M3-Embedding): BAAI paper on one embedding model that emits dense, sparse and multi-vector (late-interaction) representations, multilingual, up to 8,192 tokens.
- Source: CHESS (Talaei et al., 2024): A multi-agent text-to-SQL pipeline whose information retriever (LSH value lookup, embedded catalog) and three-stage schema selector prune large schemas, with ablations quantifying each step and showing schema linking errors fall sharply.
- Source: Chinchilla Scaling: A Replication Attempt (Besiroglu et al., 2024): Epoch AI re-fit Chinchilla’s parametric scaling law from the paper’s plots and found the original Approach 3 estimates were an optimiser artefact; the corrected fit gives about 20 tokens per parameter.
- Source: Choose a design pattern for your agentic AI system (Google Cloud): Google Cloud Architecture Center guide to 12 agent design patterns and how to choose between them.
- Source: Choosing the right model (Anthropic): Anthropic’s guide to picking a Claude model: criteria, efficiency-first vs capability-first starts, a selection matrix and eval-driven upgrades.
- Source: Chroma documentation introduction: Chroma’s overview: an Apache-2.0 retrieval database for AI apps with dense, sparse and hybrid search, metadata filtering, local and cloud deployment.
- Source: Chroma: Evaluating Chunking Strategies for Retrieval: Chroma technical report measuring chunkers at the token level; finds defaults like 800/400 are poor and 200-token chunks do well.
- Source: Claude API 2026-10-21 migration: sampling parameters removed: Anthropic’s migration guide and deprecations page recording the removal of temperature, top_p and top_k from the Claude API.
- Source: Claude API data residency docs: Anthropic’s per-request inference_geo (global or us, at 1.1x price for US-only) and immutable workspace geo for storage at rest.
- Source: Claude API Docs — Extended Thinking (manual budget): Anthropic’s reference for the legacy budget_tokens thinking mode: budget rules, interleaving beta header, caching, migration to adaptive thinking.
- Source: Claude API Docs — Steering Thinking: Anthropic’s guide to adaptive thinking: effort levels, prompt steering, cache effects, cost control and billing of thinking tokens.
- Source: Claude API Docs — Thinking: Anthropic’s reference for Claude thinking: modes per model, display, signatures, tool use, interleaving, caching, context-window rules.
- Source: Claude API overview: Anthropic’s API overview: endpoints, authentication and version headers, request size limits by endpoint and platform, response headers, pagination and tiered limits.
- Source: Claude models overview (Anthropic): Anthropic’s model catalogue: current Claude lineup, IDs, context windows, output limits, prices, cutoffs and retirement dates.
- Source: Claude web fetch tool docs: Anthropic’s reference for the server-side web_fetch tool: URL validation against exfiltration, content limits, caching, citations and dynamic filtering.
- Source: Claude web search tool docs: Anthropic’s reference for the server-side web_search tool: versions, dynamic filtering, domain and location controls, citations, errors and pricing.
- Source: Claude’s Extended Thinking (Anthropic, 2025): Anthropic’s launch post for extended thinking in Claude 3.7 Sonnet: serial vs parallel test-time compute and why the thought process is shown.
- Source: Claude’s new constitution (Anthropic, 2026): Anthropic’s January 2026 announcement of a rewritten, CC0-licensed constitution that explains reasons rather than listing rules, and is used to generate Claude’s training data.
- Source: Clavié et al. 2024: Token Pooling for Multi-Vector Retrieval: Answer.AI paper showing that clustering and mean-pooling similar document token vectors at index time cuts ColBERT storage by 50% with almost no loss.
- Source: Cloud Security Alliance research note on LiteLLM CVE-2026-42208: CSA’s analysis of the LiteLLM proxy pre-auth SQL injection: timeline, in-the-wild exploitation about 36 hours after disclosure, what attackers queried, and remediation advice.
- Source: Cloudflare AI Gateway docs (features, caching, request handling): Cloudflare’s edge AI gateway: analytics, logging, exact-match caching via headers, rate limiting, per-request timeouts and retries, and dynamic routes.
- Source: Code execution (Gemini API docs): Gemini’s built-in Python code execution tool: 30-second runtime, up to 5 self-retries, fixed libraries, matplotlib-only charts, no extra charge beyond tokens.
- Source: Code execution tool (Anthropic docs): Anthropic’s server-side code execution tool: a Linux container with Bash and file sub-tools, no internet, 5 GiB RAM, 1 CPU, 30-day containers, billed per container-hour.
- Source: Code execution with MCP (Anthropic, 2025): Anthropic engineering post: exposing MCP tools as code APIs in a sandbox cut one workflow from about 150k to 2k tokens, at the cost of running a secure sandbox.
- Source: Code Interpreter tool guide (OpenAI docs): OpenAI’s hosted Python tool: auto or explicit containers with 1g/4g/16g/64g memory tiers, a 20-minute idle expiry, and generated files cited as container_file_citation annotations.
- Source: CodeAct (Microsoft Agent Framework docs): Agent Framework’s CodeAct capability: an execute_code tool backed by Hyperlight, Monty or local executors that collapses many tool calls into one sandboxed program.
- Source: Codex agent approvals and security (OpenAI docs): Codex’s two-layer safety model: OS sandbox modes (read-only, workspace-write, danger-full-access) and approval policies, with network off by default.
- Source: Cognitive Architectures for Language Agents (Sumers et al., 2023): CoALA framework paper that maps cognitive-science memory types (working, episodic, semantic, procedural) and internal actions (retrieval, reasoning, learning) onto LLM agents.
- Source: Cohere docs: Rerank overview: Cohere documentation for its hosted reranking models and API.
- Source: Cohere embeddings documentation: Cohere’s guide to its Embed models (embed-v5.0-pro and embed-v3.0): input_type, Matryoshka dimensions, compressed embedding types and image embeddings.
- Source: Collaborative workflows (ADK docs): ADK 2.0 coordinator-with-sub-agents pattern and the chat, task and single_turn sub-agent modes.
- Source: ColPali (Faysse et al., ICLR 2025): Retrieves document pages by embedding page images with a vision-language model into multi-vector late-interaction embeddings, skipping OCR and layout parsing; introduces the ViDoRe benchmark.
- Source: Compaction (OpenAI guide): OpenAI’s guide to compacting long Responses API conversations, server-side via context_management or explicitly via /responses/compact, using opaque encrypted compaction items.
- Source: Compaction at a token threshold (Claude API docs): Reference for Anthropic’s compact_20260112 context-management edit: the API summarises the conversation inside the request that crosses an input-token trigger.
- Source: Compaction on demand (Claude API docs): Reference for Anthropic’s on-demand compaction: the application sends a separate summarise request and swaps the returned signed compaction block in for the messages it covers.
- Source: Compaction overview (Claude API docs): Anthropic’s server-side summarisation of older turns, offered on demand or at a token threshold, as an alternative to writing your own summariser.
- Source: Compliance API (Anthropic): Anthropic’s programmatic audit interface: the Activity Feed, directory, effective settings, and retrieval or deletion of chats, files, projects and session transcripts.
- Source: Composio authentication docs: Composio’s model of per-user auth for agent tools: auth configs per toolkit, connected accounts per user, sessions with an in-chat connect meta-tool, and server-side credential injection.
- Source: Configure permissions (Claude Agent SDK docs): The six-step permission evaluation order of the Claude Agent SDK, allow/deny rules, and the six permission modes.
- Source: Constitutional AI: Harmlessness from AI Feedback: Anthropic’s 2022 paper training a harmless assistant using AI-generated critiques and AI preference labels guided by a written constitution.
- Source: Constitutional Classifiers: Defending against Universal Jailbreaks: Anthropic’s January 2025 paper on input and output classifiers trained on constitution-generated synthetic data, with red-teaming and cost results.
- Source: Context editing (Claude API docs): Anthropic’s beta server-side strategies for clearing old tool results and thinking blocks, with a pre-clear warning that lets the memory tool save state.
- Source: Context Engineering for AI Agents: Lessons from Building Manus (Ji, 2025): Manus co-founder’s lessons on production agent context: KV-cache hit rate, append-only context, the file system as restorable memory, recitation and keeping errors visible.
- Source: Context Rot: How Increasing Input Tokens Impacts LLM Performance (Chroma): Chroma technical report testing 18 LLMs and showing performance falls as input grows, even on simple tasks, in ways NIAH does not reveal.
- Source: Context windows (Claude API docs): Anthropic’s documentation on what counts toward Claude’s context window, window sizes by model, thinking-token handling and overflow behaviour.
- Source: Conversation state (OpenAI guide): OpenAI’s guide to multi-turn state in the Responses API: previous_response_id chaining, the Conversations API, 30-day storage, manual replay and compaction pointers.
- Source: Conversations & Memory overview (Microsoft Agent Framework docs): AgentSession usage in Agent Framework: create, reuse, serialize and resume sessions, local vs service-managed history, and the tenant-isolation caveat for service conversation IDs.
- Source: Cormack, Clarke & Büttcher 2009: Reciprocal Rank Fusion: SIGIR 2009 short paper introducing RRF, the score-free rank fusion formula used by most hybrid search engines.
- Source: crewAI GitHub README: README of the crewAI repository: standalone framework, Crews and Flows, AMP enterprise suite, licence and requirements.
- Source: CrewAI introduction (CrewAI docs): CrewAI’s own framing of its two layers: Flows for structure and state, Crews for autonomous role-based agent teams.
- Source: Crews (CrewAI docs): CrewAI Crew attributes, kickoff variants, CrewOutput, usage metrics and replay from a task.
- Source: CSA research note, OWASP’s 2026 LLM Top 10 and Agent Control Standard: Cloud Security Alliance analysis of the 2026 OWASP LLM list and the Agent Control Standard v0.1, with a prioritised action list for security teams.
- Source: Cumulated Gain-Based Evaluation of IR Techniques (Järvelin and Kekäläinen): The 2002 ACM TOIS paper that introduced cumulated gain, DCG and normalized DCG for graded relevance judgments.
- Source: Cursor Rules (Cursor docs): Cursor’s instruction system: team, project (.cursor/rules .mdc) and user rules with always/intelligent/glob/manual application, plus nested AGENTS.md.
- Source: Custom commands (Gemini CLI docs): Gemini CLI’s TOML slash commands: file locations, namespacing, {{args}}, confirmed !{shell} injection and @{file} injection.
- Source: Custom instructions with AGENTS.md (OpenAI Codex docs): How Codex discovers, orders and size-limits AGENTS.md files (global, root-to-cwd, override and fallback names, 32 KiB default cap).
- Source: DAPO (Yu et al., 2025): ByteDance Seed / Tsinghua paper giving an open, large-scale RL recipe (DAPO) that fixes four failure modes of vanilla GRPO on long chain-of-thought training.
- Source: Data controls in the OpenAI platform: OpenAI’s API data-handling guide: no training by default, 30-day abuse-monitoring logs, Modified Abuse Monitoring and Zero Data Retention, endpoint eligibility, application-state retention and EKM.
- Source: Data, privacy, and security for Foundry Models sold by Azure: Microsoft’s statement of how Azure OpenAI and other Azure-sold models in Foundry process, store and monitor customer prompts, including Global/DataZone processing and modified abuse monitoring.
- Source: Datadog LLM Observability: OpenTelemetry GenAI ingestion: Datadog’s docs and launch post for ingesting OTel GenAI semantic-convention (v1.37+) and OpenInference spans into LLM/Agent Observability without its own SDK.
- Source: Dated Data: Tracing Knowledge Cutoffs in Large Language Models: Johns Hopkins paper showing a model’s effective knowledge cutoff often differs from its reported cutoff, and why.
- Source: Daytona documentation (sandboxes and network limits): Daytona sandboxes: sub-90 ms starts, 1 vCPU/1 GiB/3 GiB defaults, ephemeral or persistent lifecycle, and tiered egress firewall with CIDR/domain allowlists or block-all.
- Source: dbt Labs, Semantic Layer vs. Text-to-SQL: 2026 Benchmark Update: dbt Labs’ April 2026 benchmark on 11 insurance-domain questions: raw-schema text-to-SQL reached 64.5%, the Semantic Layer reached 100% within its scope, and light data modeling lifted both.
- Source: DeepEval docs (G-Eval, DAG, faithfulness, contextual precision, evaluation flow) and Confident AI pricing: Confident AI’s DeepEval documentation on how its metrics and test runs work, plus the hosted Confident AI plans.
- Source: DeepEval docs: Answer Relevancy and Contextual Recall metrics: DeepEval’s documentation of how AnswerRelevancyMetric and ContextualRecallMetric are computed, their parameters, eval modes and caveats.
- Source: DeepSeek API Docs — Thinking Mode: DeepSeek’s reference for thinking mode: enabling it, effort mapping, reasoning_content, multi-turn and tool-call rules, ignored sampling params.
- Source: DeepSeek V4 model cards (DeepSeek, Hugging Face): Model cards for DeepSeek-V4-Pro-0813 and DeepSeek-V4.1-Flash-1017: MIT-licensed MoE models with 1M context and built-in speculative decoding.
- Source: DeepSeek-R1 (DeepSeek-AI, 2025): Paper showing reasoning emerges from pure RL with rule-based rewards (R1-Zero, GRPO), plus the four-stage R1 pipeline and distillation into small models.
- Source: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model: DeepSeek paper introducing Multi-head Latent Attention, which compresses the KV cache into a latent vector.
- Source: DeepSeek-V3 Technical Report: DeepSeek-V3 (671B total / 37B active): 256 fine-grained routed experts + 1 shared, auxiliary-loss-free load balancing, node-limited routing, FP8 training and expert-parallel deployment.
- Source: DeepSeek-V3/R1 Inference System Overview: DeepSeek’s production serving write-up: cross-node expert parallelism with prefill/decode disaggregation, compute-communication overlap, EP load balancing, and per-node throughput and cost figures.
- Source: DeepSeekMath and GRPO: The 2024 DeepSeek paper that introduced Group Relative Policy Optimization, a critic-free PPO variant.
- Source: DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models: DeepSeek’s MoE design: many fine-grained routed experts plus always-on shared experts, the template later used by DeepSeek-V2/V3, Llama 4, Qwen and Kimi.
- Source: Defeating Nondeterminism in LLM Inference (Thinking Machines Lab): Thinking Machines post showing temperature-0 nondeterminism comes from batch-size-dependent kernels, and building batch-invariant kernels to fix it.
- Source: Defeating Prompt Injections by Design (CaMeL): Google DeepMind and ETH (2025): a privileged LLM writes code from the trusted query, a tool-less quarantined LLM parses untrusted data, and an interpreter enforces capability-based data-flow policies.
- Source: Defending Against Indirect Prompt Injection Attacks With Spotlighting: Microsoft (2024): mark the provenance of untrusted input by delimiting, datamarking or encoding it; cuts attack success from over 50% to under 2% on GPT models.
- Source: Define success criteria and build evaluations (Claude docs): Anthropic docs on writing measurable success criteria, eval design principles, example eval types and tips for LLM-based grading.
- Source: Deliberative Alignment (Guan et al., OpenAI, 2024): OpenAI’s method for teaching reasoning models the text of a safety spec and training them to reason over it, used for o1 and o3-mini, lowering both jailbreak success and over-refusal.
- Source: Demystifying evals for AI agents (Anthropic): Anthropic engineering post defining tasks, trials, graders and outcomes, and giving a roadmap for building agent eval suites.
- Source: Design Patterns for Securing LLM Agents against Prompt Injections: Beurer-Kellner et al. (2025): six architectural patterns that constrain an agent once it has read untrusted input, with utility/security trade-offs and ten case studies.
- Source: Detecting hallucinations in large language models using semantic entropy (Nature, 2024): Oxford paper that detects confabulations by sampling several answers, clustering them by meaning, and measuring entropy over meanings.
- Source: Dhulipala et al. 2024: MUVERA: Google Research paper that compresses a multi-vector set into one fixed-dimensional encoding so ordinary single-vector ANN can fetch candidates for MaxSim reranking.
- Source: DIN-SQL (Pourreza and Rafiei, NeurIPS 2023): Decomposes LLM text-to-SQL into schema linking, query classification, class-specific generation and self-correction; its error analysis found schema linking is the largest failure category.
- Source: Direct Preference Optimization (DPO): The 2023 Stanford paper showing RLHF’s objective can be optimised with a classification loss on preference pairs.
- Source: DiskANN: Fast Accurate Billion-point Nearest Neighbor Search on a Single Node: NeurIPS 2019 paper introducing the Vamana graph and an SSD-resident ANN index that serves a billion vectors from one 64 GB machine.
- Source: Distilling the Knowledge in a Neural Network: Hinton, Vinyals and Dean’s 2015 paper that named and formalised knowledge distillation.
- Source: DistServe: Disaggregating Prefill and Decoding for Goodput-optimized LLM Serving: OSDI 2024 paper arguing for running prefill and decode on separate GPUs, optimizing goodput under TTFT/TPOT SLOs.
- Source: Do NOT Think That Much for 2+3=? (Chen et al., 2024): Paper characterising ‘overthinking’ in o1-style models (many tokens on trivial problems) and a self-training fix that shortens reasoning without losing accuracy.
- Source: Docling Documentation: Official Docling project docs: supported input formats, export formats (Markdown, HTML, DocTags, JSON), OCR engines, VLM pipeline with GraniteDocling, chunkers, MCP server and framework integrations.
- Source: Docling Technical Report (Auer et al., 2024): IBM Research paper describing Docling’s open-source PDF conversion pipeline: PDF backends, an RT-DETR layout model trained on DocLayNet, TableFormer table structure, optional OCR, assembly, and CPU throughput numbers.
- Source: Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?: A 2024 Google study showing that SFT examples with unknown facts are learned slowly and raise hallucination.
- Source: Does RL Really Incentivize Reasoning Beyond the Base Model? (Yue et al., 2025): NeurIPS 2025 oral showing RLVR models win at pass@1 but lose to their base models at large pass@k, implying RLVR sharpens rather than expands reasoning.
- Source: Don’t Build Multi-Agents (Cognition, 2025): Cognition (Devin) argues for single-threaded agents that share full context, warning that parallel subagents make conflicting implicit decisions.
- Source: DSPy 3.5 release notes: DSPy 3.5 (2026-11-10): XMLAdapter becomes the default, LiteLLM becomes optional via the lm15 engine, ReActV2 replaces ReAct, and RLM, Flex and GEPA leave experimental.
- Source: DSPy documentation (signatures, modules, adapters, optimizers): DSPy’s official docs on declaring LM behaviour as signatures, composing modules, how adapters turn signatures into prompts, and the catalogue of prompt and weight optimizers.
- Source: DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (Khattab et al., 2023): The paper introducing DSPy: LM pipelines as text-transformation graphs of parameterized declarative modules, optimized by a compiler against a metric.
- Source: dsRAG (D-Star AI) README: Open-source retrieval engine combining semantic sectioning, contextual chunk headers (AutoContext) and relevant segment extraction, with FinanceBench and KITE results.
- Source: E2B documentation (sandboxes, internet access, secrets, security): E2B’s Firecracker-microVM sandboxes for agents: pause/resume persistence, 1h/24h run limits, IP/CIDR/domain egress rules via SNI/Host inspection, and proxy-injected secrets.
- Source: EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test: Paper on the third generation of EAGLE draft heads, which fuse multi-layer target features and predict tokens directly.
- Source: Edge et al. 2024: From Local to Global, a Graph RAG Approach: Microsoft Research paper introducing GraphRAG: LLM-built entity graph, Leiden communities and map-reduce over community summaries.
- Source: EDPB Guidelines 02/2026 on Anonymisation (draft): Draft EDPB guidelines (July 2026) replacing the 2014 test with three criteria (no isolation, no linkage, no inference) and a relative, recipient-aware notion of anonymity that explicitly covers AI model outputs.
- Source: EDPB Opinion 28/2024 on AI models and personal data: The EU data protection board’s December 2024 opinion: an AI model trained on personal data is anonymous only if extraction and query-based disclosure are insignificant; legitimate interest needs a three-step test.
- Source: Effective context engineering for AI agents (Anthropic): Anthropic Applied AI post framing context as a finite attention budget subject to context rot, with compaction, notes, sub-agents and just-in-time retrieval as remedies.
- Source: Effective harnesses for long-running agents (Anthropic, 2025): Anthropic engineering post on an initializer-agent plus incremental coding-agent harness that carries multi-session app builds across context windows.
- Source: Efficient and robust ANN search using Hierarchical Navigable Small World graphs (Malkov & Yashunin): The HNSW paper: a multi-layer proximity graph with exponentially distributed levels, a diversity-based neighbor heuristic and logarithmic search scaling.
- Source: Efficient Guided Generation for Large Language Models: Willard and Louf (2023): constrained decoding by indexing the vocabulary against a finite-state machine compiled from a regex or grammar; basis of Outlines.
- Source: Efficient Memory Management for LLM Serving with PagedAttention: SOSP 2023 paper introducing PagedAttention and the vLLM serving engine.
- Source: Elastic License 2.0 (ELv2) text: The source-available licence used by Arize Phoenix: permissive use with three limitations on managed-service offering, licence-key circumvention and notices.
- Source: Elastic: Practical BM25, Part 2: Elastic blog explaining the BM25 formula’s IDF, term-frequency saturation (k1) and length normalization (b).
- Source: Elasticsearch docs: Reciprocal rank fusion retriever: Elasticsearch reference for the RRF retriever that fuses BM25 and kNN results by rank.
- Source: Elasticsearch docs: Similarity settings: Elasticsearch reference for per-field similarity: BM25 is the default (k1 1.2, b 0.75), with DFR, DFI, IB, LM and scripted alternatives.
- Source: Envoy AI Gateway v1.0 release notes (now Agent Router): Release notes for the first GA of Envoy AI Gateway (2026-06-23): Kubernetes-native unified OpenAI-compatible API over 16 providers, provider fallback, token quotas, InferencePool and MCP routing; project later renamed Agent Router.
- Source: Equipping agents for the real world with Agent Skills (Anthropic Engineering): Anthropic’s launch essay for Agent Skills: folders of instructions, scripts and resources loaded by progressive disclosure, later published as an open standard.
- Source: Es et al. 2023: RAGAS, Automated Evaluation of Retrieval Augmented Generation: Paper introducing Ragas: reference-free, LLM-computed faithfulness, answer relevance and context relevance metrics.
- Source: EU AI Act Article 26, obligations of deployers of high-risk AI systems: What an organization that uses (rather than builds) a high-risk AI system must do: follow instructions, assign competent human oversight, check input data, monitor, keep logs at least six months, inform workers and affected people.
- Source: EU AI Act Article 99, penalties: The AI Act’s fine tiers: up to EUR 35M or 7% of worldwide turnover for prohibited practices, EUR 15M or 3% for other obligations, EUR 7.5M or 1% for misleading information.
- Source: EU AI Act implementation timeline: Date-by-date schedule of when each part of the EU AI Act applies, from entry into force (2024-08-01) to the 2030-2031 legacy deadlines, reflecting the 2026 Digital Omnibus delays.
- Source: Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents? (Gloaguen et al., 2026): ETH Zurich / LogicStar study finding that AGENTS.md-style context files do not generally raise coding-agent success and add over 20% inference cost.
- Source: Evaluating Large Language Models Trained on Code (Chen et al., 2021): The Codex/HumanEval paper, which defined pass@k with an unbiased estimator and showed repeated sampling boosts success.
- Source: Evaluation best practices (OpenAI API docs): OpenAI guide to the eval process, grader types, LLM-as-judge modes, where to add evals in agent architectures, and anti-patterns.
- Source: Event loop (ADK runtime docs): How the ADK Runner and agents cooperate through yielded events, state commits and invocations.
- Source: Exa search API docs: Exa’s reference for its own-index, embeddings-based web search API: search types and latency, content options (highlights, text, summary), filters and freshness control.
- Source: Executable Code Actions Elicit Better LLM Agents (CodeAct, Wang et al., 2024): ICML 2024 paper proposing CodeAct: agents act by emitting executable Python run in an interpreter, beating JSON/text tool-call formats by up to 20% success.
- Source: Exponential Backoff and Jitter (AWS Architecture Blog): Marc Brooker’s 2015 simulation comparing no backoff, plain exponential backoff, and Full/Equal/Decorrelated jitter; jitter cuts total calls by more than half under contention; Full and Decorrelated Jitter both beat Equal Jitter.
- Source: Extend Claude Code (Claude Code docs): Anthropic’s map of Claude Code extension points (CLAUDE.md, rules, skills, MCP, subagents, hooks, plugins) with when-to-use guidance and per-feature context cost.
- Source: Extend Claude with skills (Claude Code docs): Claude Code’s skills reference: locations and precedence, the extended frontmatter, argument substitution, shell injection, invocation control, listing budget and troubleshooting.
- Source: Extensions (Gemini CLI docs): Gemini CLI extensions package prompts, MCP servers, custom commands, themes, hooks, subagents and skills, installed from a GitHub URL.
- Source: Extracting Training Data from Large Language Models: Carlini et al. (2020/2021): querying GPT-2 recovered hundreds of verbatim training sequences, including names, phone numbers and emails; larger models leak more.
- Source: Faiss indexes (Faiss wiki): Reference table of Faiss index types with their parameters, bytes per vector and whether search is exhaustive.
- Source: Fast and Expressive LLM Inference with RadixAttention and SGLang: LMSYS blog post explaining RadixAttention’s radix-tree KV cache and LRU eviction.
- Source: Fast Inference from Transformers via Speculative Decoding: ICML 2023 paper introducing speculative decoding and the distribution-preserving speculative sampling rule.
- Source: File Search (Gemini API docs): Google’s hosted RAG tool for Gemini: persistent file search stores, chunking config, metadata filters, file_citation annotations, limits and pricing.
- Source: File search tool (OpenAI guide): OpenAI’s hosted file search tool: vector stores, file_search_call output, file_citation annotations, max_num_results and metadata filters.
- Source: Files API (Claude docs): Anthropic’s Files API: upload once and reference by file_id in document, image or container_upload blocks; workspace scoping, limits, expiry, and ZDR ineligibility.
- Source: Files API (Gemini API docs): Google’s Gemini Files API: 2 GB files, 20 GB per project, 48-hour retention, free, used when requests exceed 100 MB.
- Source: Fine-Tuning or Retrieval? Comparing Knowledge Injection in LLMs: A 2023 Microsoft study finding RAG beats unsupervised fine-tuning for injecting knowledge.
- Source: Fine-tuning vs RAG vs Prompting (Notebook): The owner’s tutorial on choosing between prompting, RAG and fine-tuning to adapt an LLM.
- Source: Firecracker (project site and NSDI 2020 paper): AWS’s open-source KVM-based microVM monitor for serverless: under 125 ms boot, under 5 MiB overhead, five emulated devices, a jailer, and production use in Lambda and Fargate.
- Source: Firecrawl docs (introduction and scrape): Firecrawl’s docs for turning websites into LLM-ready markdown or JSON: scrape, crawl, map, search, interact and agent endpoints, caching via maxAge and credit costs.
- Source: FlashAttention (Dao et al., 2022): An exact attention algorithm that tiles computation to cut GPU memory traffic, making long sequences faster and cheaper.
- Source: Flows (CrewAI docs): CrewAI Flows: @start/@listen/@router decorators, structured state, persistence, human feedback and embedding crews.
- Source: Formal et al. 2021: SPLADE v2: Follow-up to SPLADE adding max pooling, a document-only variant and distillation, with BEIR zero-shot results above BM25.
- Source: Formal, Piwowarski & Clinchant 2021: SPLADE: SIGIR 2021 short paper introducing SPLADE, a learned sparse retriever that predicts vocabulary-wide term weights with expansion and served from an inverted index.
- Source: Forward Deployed Engineer, Applied AI (Anthropic job posting): Anthropic’s 2025-26 posting for founding Applied AI FDEs: build production Claude applications in customer environments, deliver MCP servers, sub-agents and agent skills.
- Source: Forward Deployed Engineers (The Pragmatic Engineer): Gergely Orosz’s 2025 deep dive into the FDE role: its Palantir origin, how OpenAI built an FDE team, and how FDEs differ from solutions architects and consultants.
- Source: Foundry Models sold by Azure — abuse monitoring: Microsoft’s description of abuse monitoring for Azure OpenAI and other models sold by Azure: classification, pattern capture, LLM-then-human review, and the Limited Access application for modified abuse monitoring.
- Source: FP8 Formats for Deep Learning: NVIDIA/Arm/Intel paper specifying the E4M3 and E5M2 8-bit floating-point formats.
- Source: FrugalGPT (Chen, Zaharia, Zou 2023): Paper proposing LLM cascades and other strategies to cut LLM API cost while matching or beating the best single model.
- Source: G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment (Liu et al., 2023): Paper introducing G-Eval: auto-generated evaluation steps, form-filling, and probability-weighted scores for LLM judges.
- Source: Gao et al. 2022: HyDE (Precise Zero-Shot Dense Retrieval without Relevance Labels): Paper introducing Hypothetical Document Embeddings: embed an LLM-written fake answer instead of the query.
- Source: garak, the LLM vulnerability scanner (NVIDIA): NVIDIA’s open-source scanner that runs probe-and-detector plugins against an LLM endpoint and reports per-detector failure rates; v0.17.1 as of October 2026.
- Source: Gartner predicts 30% of GenAI projects abandoned after POC (2024): Gartner’s July 2024 forecast that at least 30% of generative AI projects would be dropped after proof of concept by end of 2025, and why.
- Source: Gartner predicts 40%+ of agentic AI projects canceled by 2027 (2025): Gartner’s June 2025 forecast that over 40% of agentic AI projects will be canceled by end of 2027, with a warning about ‘agent washing’.
- Source: GCP Services & IaC (Notebook): The owner’s tutorial on core GCP services, the project/IAM hierarchy and Terraform, placing Vertex AI as GCP’s ML platform.
- Source: GDPR Article 4, definitions: The legal definitions that decide whether prompts, logs and embeddings are regulated: personal data, processing, pseudonymisation, controller and processor.
- Source: Gemini API abuse monitoring policy: Google’s abuse-monitoring section of the Gemini API usage policies: prompts, context and outputs kept 55 days, human review of flagged content, no stated exemption.
- Source: Gemini API Additional Terms of Service: Google’s Gemini API terms: Unpaid Services may be used for training and human review, Paid Services are processed under Google’s processor DPA, EEA/UK/Swiss end users require Paid Services, and grounding data is stored 30 days.
- Source: Gemini API Context Caching and Pricing Docs: Google Gemini API documentation on implicit and explicit context caching, plus the pricing page’s cache and batch rates.
- Source: Gemini API data logging and sharing: Google’s AI Studio project-logs feature: on by default for Interactions API calls and for newer projects, configurable 7 to 55 day retention, separate from abuse-monitoring logs, with opt-in dataset sharing.
- Source: Gemini API Docs — Thinking: Google’s reference for Gemini thinking: thinking levels per model, thought summaries, thought signatures and billing.
- Source: Gemini API Function Calling Docs: Google Gemini API documentation on function calling: declarations, modes (AUTO/ANY/NONE/VALIDATED), parallel and compositional calls, thought signatures.
- Source: Gemini API Grounding with Google Search docs: Google’s guide to the google_search tool in the Gemini API: model-driven queries, url_citation spans, search suggestions and per-query pricing.
- Source: Gemini API pricing (Google): Google’s Gemini API prices by tier (Standard, Batch/Flex, Priority), with scheduled increases, caching and free-tier rules.
- Source: Gemini API Streaming Interactions Docs: Google Gemini API documentation on SSE streaming in the Interactions API using a step-based event model.
- Source: Gemini API Structured Outputs Docs: Google Gemini API documentation on JSON Schema structured outputs: schema compiled to a state machine, unenforced keywords and complexity limits.
- Source: Gemini API — Rate limits: Gemini Developer API limits: RPM/TPM/RPD per project (not per key), midnight-Pacific RPD reset, spend-based usage tiers, a rolling 10-minute spend limit, Priority inference limits, and Batch enqueued-token caps.
- Source: Gemini Batch API Docs: Google Gemini API documentation on the Batch API: 50% cost, 24-hour target, inline vs JSONL file input, job states.
- Source: Gemini Deep Research agent docs: Google’s guide to the Deep Research agent in the Interactions API: agent IDs, background execution, streaming, collaborative planning and tools.
- Source: Gemini Developer API — Zero data retention: Google’s page on reaching zero data retention with the Gemini Developer API: no single switch, a per-feature checklist, and a pointer to Vertex AI for guaranteed ZDR.
- Source: Gemini Enterprise Agent Platform — Agent Runtime Overview (formerly Vertex AI Agent Engine): Google’s managed agent hosting and its companion services: Sessions, Memory Bank, Code Execution, Example Store, evaluation and observability.
- Source: Gemini Enterprise Agent Platform — Release Notes (2026): Google’s 2026 release notes, including the April 22, 2026 rebrand of Vertex AI and its agent, RAG and model services.
- Source: Gemini Image Understanding Docs: Google Gemini API documentation on image input: inline vs Files API, 258-token tiles, media_resolution, normalized bounding boxes and segmentation.
- Source: Gemini Live API Docs: Google’s overview of the Gemini Live API: low-latency streaming voice and vision over a stateful WebSocket, audio formats, barge-in, tools and ephemeral tokens.
- Source: Gemini models (Google AI Studio docs): Google’s Gemini API model list: Gemini 4 Carbon, Argon and Hydrogen, Gemini 3.x Flash/Flash-Lite, specialised models and deprecations.
- Source: GEMINI.md context files (Gemini CLI docs): How Gemini CLI loads hierarchical GEMINI.md files (global, workspace ancestors, just-in-time), supports @imports, and can be pointed at AGENTS.md.
- Source: Gemma 4 model overview and card (Google): Google’s Gemma 4 docs and model card: five sizes from on-device to 31B, dense and MoE, up to 256K context, Apache 2.0.
- Source: generateContent / GenerationConfig (Gemini API reference): Google’s API reference for Gemini generation settings: temperature, topP, topK, stop sequences, penalties, seed and logprobs.
- Source: Generative Agents: Interactive Simulacra of Human Behavior (Park et al., 2023): Paper introducing the memory stream with recency/importance/relevance retrieval scoring and importance-triggered reflection, shown by ablation to drive believable agent behaviour.
- Source: GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (Agrawal et al., 2025): GEPA optimizes prompts by reflecting in natural language on sampled trajectories and keeping a Pareto frontier of candidates; it reports beating GRPO with up to 35x fewer rollouts.
- Source: GitHub Copilot cloud (coding) agent docs: overview and risks: GitHub’s docs for its asynchronous Copilot agent: Actions-based ephemeral environments, firewall, copilot/ branches, human-only PR approval and other mitigations.
- Source: GitHub MCP exploited (Willison link post): Willison’s May 2025 write-up of Invariant Labs’ attack where a malicious public GitHub issue made an agent leak private repo names via a public pull request.
- Source: Give Claude custom tools (Claude Agent SDK docs): How to define custom tools for the Claude Agent SDK as in-process MCP servers, with annotations, permissions and error handling.
- Source: Give Me FP32 or Give Me Death? Challenges and Solutions for Reproducible Reasoning (Yuan et al., 2025): Paper showing that batch size, GPU count and GPU type change greedy-decoding results under BF16, with up to 9% accuracy spread on reasoning models; FP32 nearly eliminates it and LayerCast is proposed as a cheaper middle ground.
- Source: Google Cloud HIPAA compliance: Google Cloud’s HIPAA page: how to accept the BAA, which products it covers (including Gemini Enterprise Agent Platform and Generative AI on Agent Platform), and the rule against PHI in pre-GA offerings.
- Source: Google: A2A, a New Era of Agent Interoperability: Google’s April 2025 launch post for the Agent2Agent protocol: five design principles, client/remote agent model and positioning next to MCP.
- Source: GPQA: A Graduate-Level Google-Proof Q&A Benchmark (Rein et al., 2023): 448 expert-written biology, physics and chemistry questions that skilled non-experts with web access mostly fail.
- Source: GPT Semantic Cache (Regmi & Pun, 2024): Reports that caching query embeddings in Redis and serving similar queries from cache cut LLM API calls by up to 68.8% with high precision on their test categories.
- Source: gpt-oss-120b & gpt-oss-20b Model Card (arXiv): OpenAI’s technical model card for gpt-oss: per-component parameter counts, 128/32 experts with top-4 routing, and MXFP4-quantised expert weights that fit 120b on one 80GB GPU.
- Source: gpt-oss-120b model card (OpenAI, Hugging Face): Model card for OpenAI’s Apache 2.0 open-weight reasoning models gpt-oss-120b and gpt-oss-20b.
- Source: gpt-oss-safeguard model card: Hugging Face card for OpenAI’s open-weight gpt-oss-safeguard 20b/120b reasoning models that classify content against a policy supplied at inference.
- Source: GPTCache usage documentation: GPTCache’s library docs: Config similarity_threshold, temperature-driven cache skipping, cache_skip, sessions, and pluggable scalar/vector stores with LRU or FIFO eviction.
- Source: GPTCache, an open-source semantic cache for LLM applications (Fu & Feng, 2023): Describes GPTCache’s embedding-based semantic cache architecture and early hit/false-hit measurements, plus why negative hits limit production use.
- Source: GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers: ICLR 2023 paper on one-shot 3-4 bit weight quantization using approximate second-order information.
- Source: GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints: EMNLP 2023 paper introducing grouped-query attention as a middle ground between MHA and MQA.
- Source: Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack: Russinovich, Salem and Eldan (Microsoft, 2024): a benign-looking conversation that escalates step by step, referencing the model’s own replies, jailbreaks frontier models.
- Source: Group Sequence Policy Optimization (GSPO): Qwen team’s 2025 RL algorithm that moves GRPO’s importance ratio and clipping from the token level to the sequence level, stabilising MoE RL.
- Source: GroUSE, a benchmark to evaluate evaluators in grounded QA (Muller et al., 2024): Meta-evaluation of LLM judges for RAG answers: 144 unit tests over 7 failure modes show automated RAG evaluation frameworks miss failures even with GPT-4.
- Source: Guardrails (OpenAI Agents SDK docs): Input, output and tool guardrails in the OpenAI Agents SDK, tripwire semantics, parallel vs blocking execution, and gaps.
- Source: Guardrails AI library and docs: README and concept docs for the open-source Guardrails AI Python library: Guard objects, Hub validators, on_fail actions and the OpenAI-compatible server.
- Source: Guiding LLMs The Right Way (DOMINO): Beurer-Kellner, Fischer and Vechev (ETH Zurich, 2024): naive constrained decoding misaligns subword tokens with grammar boundaries and costs accuracy; DOMINO fixes this with near-zero overhead.
- Source: gVisor documentation (overview and platforms): gVisor is a Go user-space application kernel (Sentry plus Gofer) that intercepts container syscalls via systrap or KVM platforms, trading syscall overhead and compatibility for isolation.
- Source: Günther et al. 2024: Late Chunking: Jina AI paper that embeds the whole document first and pools token vectors per chunk afterwards, so chunks keep document context.
- Source: Hamburg DPA discussion paper, LLMs and personal data: The Hamburg data protection commissioner’s July 2024 position that LLM weights do not store personal data, so GDPR rights attach to the AI system’s inputs and outputs; a position later overtaken by EDPB Opinion 28/2024.
- Source: Handoff orchestration (Microsoft Agent Framework docs): Agent Framework’s mesh-topology handoff orchestration: handoff rules, context broadcast, interactive vs autonomous mode, and a stated handoff vs agent-as-tools contrast.
- Source: Handoffs (OpenAI Agents SDK docs): How OpenAI Agents SDK handoffs work: transfer tools, the handoff() options, input filters and nested history.
- Source: Harness design for long-running application development (Anthropic, 2026): Anthropic Labs post on a planner / generator / evaluator harness for multi-hour app builds, with cost data and an ablation showing which parts stopped paying off on a newer model.
- Source: Harness engineering: leveraging Codex in an agent-first world (OpenAI, 2026): OpenAI post on building an internal product with Codex writing essentially all code, recasting the engineer’s job as designing the environment, intent and feedback loops.
- Source: Harness Engineering: The Work Around the Model: Owner blog post arguing that agent = model + harness, mapping the harness layers and distilling lessons from Anthropic, OpenAI and LangChain write-ups.
- Source: Harness vs Loop vs Graph Engineer (Notebook tutorial): The owner’s tutorial placing graph, loop and harness engineering on a control-versus-autonomy dial, with coding agents as the canonical harness case.
- Source: Help Net Security, OWASP 2026 LLM Top 10: The model will be fooled: News report on the 2026 OWASP LLM Top 10 with the project leads’ methodology: 75% practitioner vote, 25% incident data from 6,639 incidents.
- Source: Holistic Agent Leaderboard (Kapoor et al., 2025): Paper introducing HAL, a standardized parallel harness that ran 21,730 agent rollouts across 9 models and 9 benchmarks and inspected logs for hidden misbehaviour.
- Source: Hooks (Gemini CLI docs): Gemini CLI lifecycle hooks (BeforeAgent, BeforeModel, BeforeToolSelection, BeforeTool, etc.) configured in settings.json, with exit-2 blocking and fingerprint-based trust.
- Source: Hooks (OpenAI Codex docs): Codex lifecycle hooks: events, hooks.json/config.toml locations, exit-2 blocking, JSON decisions, hash-based trust and managed hooks.
- Source: Hooks reference (Claude Code docs): Claude Code’s deterministic lifecycle handlers: ~30 events, five handler types, matcher syntax, exit-code and JSON decision semantics, timeouts and trust controls.
- Source: Hosting the Agent SDK (Claude Code docs): Production hosting guidance for the Claude Agent SDK: subprocess model, session patterns, resources, observability, multi-tenant isolation and known limits.
- Source: How Claude Code works (Anthropic docs): Architecture overview of Claude Code: the gather-act-verify loop, built-in tool categories, execution environments, sessions, compaction, checkpoints and permission modes.
- Source: How Claude remembers your project (Claude Code memory docs): Reference for CLAUDE.md locations, load order, imports, path-scoped rules, managed policy files, auto memory and AGENTS.md interop in Claude Code.
- Source: How continuous batching enables 23x throughput in LLM inference: Anyscale engineering blog benchmarking static vs continuous batching and vLLM.
- Source: How Embedding Models Work (Notebook tutorial): The owner’s tutorial on embedding models: contrastive training, pooling, similarity metrics, model choice and common traps.
- Source: How the agent loop works (Claude Agent SDK docs): Claude Agent SDK documentation of turns, message types, tool execution, budgets, effort, context accumulation, compaction and result subtypes.
- Source: How to make your completions outputs consistent with the new seed parameter (OpenAI Cookbook): OpenAI Cookbook notebook introducing the beta seed parameter and system_fingerprint, and what ‘mostly deterministic’ means for the API.
- Source: How we built our multi-agent research system (Anthropic): Anthropic’s engineering write-up of Claude Research: an orchestrator-worker system with a lead agent, parallel search subagents and a citation agent.
- Source: Huang et al. 2025: Deep Research Agents survey: Survey defining deep research agents and classifying them by workflow, agent composition, planning and API- versus browser-based information access.
- Source: Hugging Face Datasets docs: Arrow and Streaming: Datasets library docs on its memory-mapped Apache Arrow cache and on streaming mode (IterableDataset, shuffle buffers, sharding, resumable state).
- Source: Hugging Face Hub docs: Gated models: How access-request gating works on the Hub: automatic vs manual approval, custom form fields, token-authenticated downloads, EU restriction.
- Source: Hugging Face Hub docs: GGUF: Hub documentation for the GGUF file format and its quantization types with bits-per-weight.
- Source: Hugging Face Hub docs: Model Cards: How Hub model cards work: README.md with YAML metadata for license, base_model and relation, datasets, pipeline_tag, library_name and eval results.
- Source: Hugging Face Inference Endpoints docs: Hugging Face’s managed dedicated deployment service: Hub model + chosen engine (vLLM, SGLang, TGI, llama.cpp, TEI) packaged as a container with autoscaling and scale-to-zero.
- Source: Hugging Face Inference Providers docs: Hugging Face’s serverless router to partner inference providers behind one token and an OpenAI-compatible endpoint, with :fastest/:cheapest/:preferred routing.
- Source: Hugging Face PEFT docs: overview and Quicktour: PEFT library overview and quicktour: LoraConfig, get_peft_model, 0.04% trainable params on Llama-3.2-1B, ~6 MB adapters, AutoPeftModel loading and multiple adapters.
- Source: Hugging Face PEFT — LoRA Developer Guide: The PEFT library’s guide to LoraConfig options, LoRA variants and adapter merging.
- Source: Hugging Face Transformers: Tokenization algorithms: Documentation page explaining BPE, byte-level BPE, Unigram, WordPiece and SentencePiece with worked examples.
- Source: Hugging Face TRL Documentation: Documentation for TRL, Hugging Face’s post-training library (SFT, DPO, GRPO, reward modelling, distillation).
- Source: huggingface/smolagents README (GitHub): smolagents repository README: Apache-2.0, ~1,000-line agents.py, code actions using about 30% fewer steps, executor options, and the warning that the local executor is not a security sandbox.
- Source: huggingface_hub guide: Download files from the Hub: The huggingface_hub client’s download APIs: hf_hub_download, snapshot_download with patterns and revisions, the cache, hf CLI, dry-run and Xet-based transfers.
- Source: Human-in-the-loop (OpenAI Agents SDK docs): OpenAI Agents SDK tool-approval flow: needs_approval, run interruptions, RunState approve/reject with sticky decisions, JSON serialization for long waits, and security guidance.
- Source: Human-in-the-loop middleware (LangChain docs): LangChain’s HumanInTheLoopMiddleware: per-tool interrupt policies, approve/edit/reject/respond decisions, batched vs per-call interrupts, and the checkpointer requirement.
- Source: Humanity’s Last Exam (Phan et al., 2025): 2,500-question expert-written multimodal benchmark at the frontier of human knowledge, reporting accuracy and calibration error.
- Source: Hybrid LLM, cost-efficient and quality-aware query routing (Ding et al., ICLR 2024): Paper routing each query to a small or large model by predicted difficulty, with a quality knob tunable at test time; up to 40% fewer large-model calls without quality loss.
- Source: IBM Granite-Docling Announcement: IBM’s September 2025 release of Granite-Docling-258M, a small Apache-2.0 vision-language model that converts document pages to DocTags in one pass, succeeding SmolDocling.
- Source: Idempotent requests (Stripe API reference): Stripe’s reference design for idempotency keys: client-generated keys, saved first response, parameter-mismatch errors, 24-hour pruning, and which requests need keys.
- Source: Identifying and Scaling AI Use Cases (OpenAI): OpenAI’s 2025 business guide: where to look for AI use cases, the six use-case primitives, and the impact/effort prioritisation matrix used by its customer teams.
- Source: Identity Assertion JWT Authorization Grant (IETF draft): IETF OAuth WG draft defining the ID-JAG: an IdP-signed JWT, obtained by token exchange, that a client presents via the JWT bearer grant to another domain’s authorization server.
- Source: illuin-tech/colpali repository (ColVision models): Reference repo for ColPali-family visual retrievers: model table (ColPali, ColQwen2/2.5, ColSmol, community ColQwen3/3.5 variants) with ViDoRe scores, licenses, token pooling, and its deprecation in favour of Sentence Transformers.
- Source: Improving agent with semantic search (Cursor, 2025): Cursor’s report that a custom, agent-trace-trained code embedding model plus grep improves agent accuracy by 12.5% on average, most on large codebases.
- Source: Inference hooks (Anthropic): Claude Enterprise feature that sends each governed prompt (and optionally tool calls) to the customer’s AI security server for an allow or deny verdict before inference.
- Source: Inngest docs: how functions are executed, step.waitForEvent and usage limits: Inngest’s step-memoization execution model (checkpointing vs standard orchestration), event-based waits for approvals, and its published limits on steps, state and run length.
- Source: Inspect documentation (UK AI Security Institute): Docs for Inspect, an open-source Python eval framework built on tasks, datasets, solvers and scorers, with agent and sandbox support.
- Source: Instructor Documentation and README: Instructor (567-labs) docs: Pydantic response_model on any provider via from_provider, modes (TOOLS, JSON_SCHEMA, MD_JSON…), validation-error reasking, Tenacity retry policies, partial/iterable streaming and hooks.
- Source: Interactions API (Gemini API docs): Google’s Interactions API: step-based requests and responses, server-side state via previous_interaction_id, retention windows, background execution, agents such as Deep Research, and its relation to legacy generateContent.
- Source: Intercept and control agent behavior with hooks (Claude Agent SDK docs): In-process hook callbacks for the Claude Agent SDK: event list by language, HookMatcher registration, output fields, decision precedence, async outputs, onFailure fail-closed mode and per-event timeout behaviour.
- Source: Introducing Meta Llama 3 (Meta AI blog): Meta’s Llama 3 launch post describing a standard decoder-only transformer with a 128K-token vocabulary and grouped-query attention.
- Source: Introducing smolagents (Hugging Face blog): Launch post for smolagents, the successor to transformers.agents, arguing for code agents and showing open models competitive with closed ones on the authors’ agent benchmark.
- Source: Introducing SWE-bench Verified (OpenAI, 2024): OpenAI’s human-validated 500-task subset of SWE-bench, built to remove underspecified issues and unfair tests.
- Source: Introduction to Information Retrieval, ch. 8: Evaluation in IR (Manning, Raghavan, Schütze): The standard textbook chapter on IR evaluation: precision, recall, MAP, precision@k, R-precision, NDCG, pooling and assessor agreement.
- Source: Introduction to the Deep Research API (OpenAI Cookbook): OpenAI cookbook recipe for the o3/o4-mini deep research models: background mode, tools, output items and the clarify-then-rewrite prompt pattern.
- Source: Inverse Scaling in Test-Time Compute (Gema et al., 2025): Paper finding that longer reasoning can lower accuracy on some tasks, and naming five distinct failure modes across Claude and o-series models.
- Source: ISO 42001 requirements explained (Konfirmity): Secondary walk-through of ISO/IEC 42001 clauses 4-10, the Statement of Applicability, the AI system impact assessment (6.1.4, 8.4) and the 38 Annex A reference controls.
- Source: ISO/IEC 42001 FAQs (AWS): AWS’s FAQ on its accredited ISO/IEC 42001:2023 certification and the point that customers are not certified by association.
- Source: ISO/IEC 42001:2023 offering (Microsoft Compliance): Microsoft’s explainer of ISO/IEC 42001 AI management systems and its certified AI services (Foundry, Copilot, GitHub Copilot and others), and how customers can use the certificate.
- Source: Jailbreaking Black Box Large Language Models in Twenty Queries (PAIR): Chao et al. (2023): an attacker LLM iteratively refines semantic jailbreak prompts against a black-box target, often in under 20 queries.
- Source: Jailbroken: How Does LLM Safety Training Fail?: Wei, Haghtalab and Steinhardt (2023): jailbreaks exploit competing objectives and mismatched generalization; scaling alone will not fix them.
- Source: Jina Reader (r.jina.ai / s.jina.ai): Jina AI’s Reader API that converts any URL into LLM-friendly markdown by URL prefix, plus a search endpoint, control headers, ReaderLM-v2 and token-based pricing.
- Source: JSONSchemaBench (Generating Structured Outputs from Language Models): Geng et al. (Jan 2025): benchmark of 10K real-world JSON schemas comparing Guidance, Outlines, llama.cpp, XGrammar, OpenAI and Gemini on efficiency, schema coverage and output quality.
- Source: Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (Zheng et al.): NeurIPS 2023 paper introducing MT-Bench and quantifying LLM-judge position, verbosity and self-enhancement biases.
- Source: Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (Ye et al., 2024): Paper defining 12 LLM-judge biases and the CALM framework that measures each by perturbing answers.
- Source: Kamphuis et al. 2020: Which BM25 Do You Mean?: ECIR 2020 reproducibility study finding no significant effectiveness differences among eight BM25 variants, including Lucene’s lossy length encoding.
- Source: Khattab & Zaharia 2020: ColBERT: SIGIR 2020 paper introducing late interaction: per-token embeddings matched cheaply at query time.
- Source: Kimi K3.1 Pro model card (Moonshot AI, Hugging Face): Model card and licence for Kimi K3.1 Pro: a 2.8T/104B-active multimodal MoE with 1M context and a revenue-gated licence.
- Source: Knowledge Distillation (Notebook): The owner’s tutorial on teacher-student distillation, from Hinton’s soft targets to LLM-era sequence and reasoning distillation.
- Source: Kong AI Gateway docs and AI Proxy Advanced plugin: Kong’s AI gateway (2.x entity model, formerly AI Proxy plugins): load-balancing algorithms incl. semantic and lowest-latency, failover criteria, circuit breaking, semantic cache, prompt guards and PII sanitising.
- Source: Kong AI Semantic Cache plugin docs: Kong’s enterprise gateway plugin for semantic response caching on Redis/Valkey or pgvector, with Cache-Control-driven expiry, X-Cache-Status headers and documented operational limits.
- Source: LangChain agents, middleware and structured output docs: Official LangChain v1 docs for create_agent, the middleware hook model, the built-in middleware catalogue, and ToolStrategy/ProviderStrategy structured output.
- Source: LangChain and LangGraph agent frameworks reach v1.0: LangChain’s 22 Oct 2025 announcement of LangChain 1.0 (create_agent, middleware, content blocks) and LangGraph 1.0 (stable durable runtime).
- Source: LangChain models, tools and retrieval docs: Official LangChain v1 docs for the chat model interface (init_chat_model, bind_tools, with_structured_output), tool definition with ToolRuntime, and retrieval building blocks.
- Source: LangChain multi-agent docs (overview and handoffs): LangChain v1 docs comparing subagents, handoffs, skills and router patterns, and showing handoffs as state-updating tools that return Command.
- Source: LangChain MultiQueryRetriever (langchain_classic): LangChain’s reference implementation of multi-query retrieval: an LLM writes 3 rephrasings, each is retrieved, and the unique union is returned.
- Source: LangChain SelfQueryRetriever and query constructor (langchain_classic): LangChain’s self-querying retriever: an LLM turns a natural-language question into a semantic query string plus a structured metadata filter, which a per-store translator converts to native filter syntax.
- Source: langchain-core Runnable implementation (runnables/base.py): The source code and docstrings of the Runnable base class that defines LCEL composition, batching, streaming, retry and fallback semantics.
- Source: Langfuse evaluation, experiments and CI/CD documentation: Langfuse docs on scores, datasets, run_experiment, managed LLM-as-judge evaluators and the experiment GitHub Action that gates PRs on regressions.
- Source: Langfuse observability documentation: Langfuse’s OTel-based data model (traces, observations, sessions), observation types, usage/cost inference rules and self-hosting architecture.
- Source: Langfuse pricing page and v4 changelog: Langfuse Cloud plan prices and limits as of 2026-11, and the 2026 v4 change to a single denormalised observations table.
- Source: Langfuse prompt management documentation: Langfuse docs on prompt types, immutable versions, labels, protected labels, config, composability, client-side caching, trace linking and prompt webhooks.
- Source: LangGraph Graph API, Functional API, streaming and subgraph docs: Official LangGraph docs on StateGraph state/reducers, Pregel super-steps, edges, Send and Command, recursion limit, the Functional API, stream modes and subgraphs.
- Source: LangGraph memory documentation: LangGraph’s concept and how-to docs on thread-scoped short-term memory (checkpointers) and cross-thread long-term memory (namespaced stores), memory types, and history trimming and summarisation.
- Source: LangGraph persistence, checkpointers and interrupts docs: Official LangGraph docs on threads, checkpoints, StateSnapshot, time travel and forking, durability modes, serializers and encryption, stores, and interrupt()/Command(resume) semantics.
- Source: LangSmith evaluation concepts and SDK docs: Official LangSmith docs on datasets, examples, splits, versions, experiments, evaluator types, offline vs online evaluation, and the evaluate() SDK call.
- Source: LangSmith observability documentation: LangSmith’s trace data model (runs, traces, threads, feedback), run schema, OTLP ingestion, sampling, retention and plan pricing.
- Source: LangSmith prompt engineering and prompt management docs: LangSmith’s commit-and-tag prompt versioning, push/pull SDK calls, model bundling, template formats and stale-while-revalidate prompt cache.
- Source: LangSmith tracing SDK, OTel export and deployment pricing: LangSmith docs on @traceable, wrap_openai, trace/RunTree and flushing, OTel environment variables, plus the pricing page’s plans and Agent Server metering and the 2025 product rename.
- Source: Language Agent Tree Search (LATS, Zhou et al., 2023): ICML 2024: Monte Carlo Tree Search over ReAct-style trajectories, with LM value functions, self-reflection and environment feedback.
- Source: Language Model Tokenizers Introduce Unfairness Between Languages (Petrov et al., 2023): Shows that the same text costs very different numbers of tokens in different languages, affecting price, latency and context capacity.
- Source: Language Models are Few-Shot Learners (GPT-3): Brown et al. (2020): a 175B-parameter model performs many tasks from in-context demonstrations alone, with no gradient updates.
- Source: Language Models Don’t Always Say What They Think: Turpin et al. (2023): chain-of-thought explanations can rationalise answers driven by hidden biasing features without mentioning them.
- Source: Large Language Models are not Fair Evaluators (Wang et al., 2023): Paper showing LLM-judge rankings flip with answer order, and proposing evidence, position-balancing and human-in-the-loop calibration.
- Source: Large Language Models are Zero-Shot Reasoners: Kojima et al. (2022): appending "Let’s think step by step" elicits chain-of-thought reasoning without any exemplars.
- Source: Large Language Models Can Accurately Predict Searcher Preferences (Thomas et al.): Microsoft/Bing study showing GPT-4 relevance labels, with a prompt tuned against real searcher preferences, match or beat crowd labellers at a fraction of the cost.
- Source: Latency optimization (OpenAI guide): OpenAI’s seven principles for cutting LLM application latency, from smaller models and fewer output tokens to parallelism, streaming and not calling an LLM at all.
- Source: Learning to Reason with LLMs (OpenAI, 2024): OpenAI’s o1 announcement: RL-trained chain of thought, with performance scaling in both train-time RL compute and test-time thinking; raw CoT hidden.
- Source: Length-Controlled AlpacaEval (Dubois et al., 2024): Paper debiasing an LLM auto-evaluator for length by regression, raising correlation with Chatbot Arena from 0.94 to 0.98.
- Source: Lessons From Red Teaming 100 Generative AI Products: Microsoft AI Red Team (Jan 2025): eight lessons from 100+ engagements, notably that simple attacks beat gradient methods, red teaming is not benchmarking, and humans remain essential.
- Source: Let Me Speak Freely? (Format Restrictions Study): Tam et al. (2024): format restrictions such as JSON mode can measurably degrade LLM reasoning, more so the stricter the constraint.
- Source: Letta documentation, MemFS, memory and dreaming, platform overview: Letta’s current docs: git-backed MemFS memory as Markdown files, dreaming subagents, cloud memory size limits, and the product line (Letta Code, Agent SDK, App Server, Cloud).
- Source: Letta documentation, memory blocks and archival memory: Letta’s docs on in-context memory blocks (label, description, value, limit, read_only, shared), searchable archival memory and conversation search.
- Source: Letta documentation, self-hosting with Docker (deprecated): Letta’s reference page for the Docker server: letta/letta image on port 8283, Postgres with pgvector, env vars, password protection and tool sandboxing; marked no longer actively supported.
- Source: Lewis et al. 2020: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks: The NeurIPS 2020 paper that named RAG: a seq2seq generator conditioned on passages from a dense Wikipedia index.
- Source: Lies, Damn Lies, & Statistics: Is Mem0 Really SOTA in Agent Memory? (Zep, 2025): Zep’s rebuttal of the Mem0 paper: argues LoCoMo is too short and noisy, says Mem0 misconfigured Zep, and reports a corrected Zep LoCoMo J score of 75.14%.
- Source: Lifecycle hooks reference (OpenAI Agents SDK, Python): API reference for RunHooks and AgentHooks in the OpenAI Agents SDK: observational callbacks for agent, LLM, tool and handoff events that return None.
- Source: LIMA: Less Is More for Alignment: Meta’s 2023 paper showing 1,000 curated SFT examples can align a 65B model.
- Source: LiteLLM blog: CVE-2026-42208, SQL injection in LiteLLM Proxy: LiteLLM’s own post on a pre-authentication SQL injection in the proxy’s API-key verification path, affecting v1.81.16-v1.83.6 and fixed in v1.83.7.
- Source: LiteLLM docs: budgets and rate limits (proxy users, teams, keys): LiteLLM proxy’s budget hierarchy (global, team, team member, internal user, virtual key, end user, per model) and rpm/tpm/parallel-request limits, with reset windows and fail-open/fail-closed behaviour.
- Source: LiteLLM docs: callbacks, custom callbacks, proxy logging: How LiteLLM emits logs: string-named integrations via success/failure callbacks, CustomLogger hook classes, the StandardLoggingPayload, OpenTelemetry export, and message-redaction switches on the proxy.
- Source: LiteLLM docs: getting started, input params, proxy config.yaml: LiteLLM’s docs on the SDK/proxy split, provider-prefixed model names, OpenAI-param translation and drop_params, and the proxy’s config.yaml (model_name aliases vs litellm_params.model).
- Source: LiteLLM GitHub repository README: BerriAI’s repository front page: LiteLLM as an open-source AI gateway with a Python SDK and a proxy server, an enterprise tier under a commercial licence, and -stable Docker images that pass load tests.
- Source: LiteLLM GitHub security advisories list: BerriAI’s published GitHub security advisories for LiteLLM; as of November 2026 the first page shows ten advisories from May to November 2026, four of them critical.
- Source: LiteLLM proxy (AI Gateway) docs: overview, virtual keys, caching: LiteLLM’s self-hosted OpenAI-compatible gateway: virtual keys with budgets and rate limits backed by Postgres, spend tracking, and exact or semantic response caches.
- Source: LiteLLM Router docs (routing, retries, fallbacks, cooldowns): LiteLLM’s reference for load-balancing strategies, per-error retry policies, fallback chains and deployment cooldowns in its Router and proxy.
- Source: LiteLLM security update, suspected supply-chain incident (March 2026): LiteLLM’s incident post: PyPI releases 1.82.7 and 1.82.8 shipped a credential stealer after a CI/CD compromise; the pinned proxy Docker image was not affected.
- Source: LiveBench: A Challenging, Contamination-Limited LLM Benchmark (White et al., 2024): Benchmark that refreshes questions monthly from recent sources and scores against objective ground truth without LLM judges.
- Source: LiveKit, Using a Transformer to Improve End of Turn Detection: LiveKit’s 2024 post introducing a 135M-parameter text-based end-of-utterance model that modulates the VAD silence timeout, cutting unintended interruptions by 85%.
- Source: Llama 4 Community License and model page (Meta): Meta’s Llama 4 licence (700M MAU clause, Built with Llama, naming rules) and the Llama 4 Scout/Maverick model page.
- Source: Llama Guard 4 model card: Hugging Face model card for Meta’s 12B multimodal Llama Guard 4 safety classifier: MLCommons S1-S14 categories, output format and measured recall/FPR.
- Source: Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations: Meta’s December 2023 paper introducing Llama Guard, an instruction-tuned Llama 2 7B that classifies prompts and responses against a taxonomy given in the prompt.
- Source: Llama Prompt Guard 2 model card: Model card for Meta’s 86M and 22M DeBERTa-based binary classifiers that flag prompt injections and jailbreaks.
- Source: llama.cpp README, llama-server and llama-quantize docs: ggml-org’s docs for llama.cpp: backends, 1.5-8 bit quantization, hybrid CPU+GPU offload, llama-server flags/endpoints, and GGUF conversion and quantization.
- Source: LlamaIndex framework documentation (RAG, indexes, query engines, agents): LlamaIndex’s official Python docs on its RAG stages, Document/Node model, index types, retrievers, node postprocessors, response modes, ingestion pipeline and FunctionAgent.
- Source: LlamaIndex retrieval evaluation metrics (source code): How llama-index-core implements HitRate, MRR, Precision, Recall, AveragePrecision and NDCG for retriever evaluation, including non-obvious normalisation choices.
- Source: LlamaIndex Workflows documentation: Docs for LlamaIndex Workflows: event-driven async steps, Context state store, human-in-the-loop events, and snapshot/restore for durable runs.
- Source: LlamaParse Tiers and Pricing Docs: LlamaIndex documentation for LlamaParse v2: the Fast, Cost-effective, Agentic and Agentic Plus parse tiers, dated version pinning, caching, and per-page credit pricing as of 2026-11.
- Source: llguidance (GitHub README): Microsoft’s guidance-ai/llguidance README: a Rust constrained-decoding engine that computes token masks on the fly from a CFG using an Earley parser over a derivative-based regex lexer and a token trie.
- Source: LLGuidance: Making Structured Outputs Go Brrr: Moskal, Nori, Cooper and Huynh (Microsoft, June 2025): how llguidance computes token masks in ~50 µs without precomputation, and the hidden costs of precomputing engines.
- Source: LLM Evaluators Recognize and Favor Their Own Generations (Panickssery et al., 2024): Paper linking an LLM’s ability to recognise its own text to its tendency to rate that text higher.
- Source: LLM Inference Performance Engineering: Best Practices: Databricks/MosaicML guide to inference metrics (TTFT, TPOT, MBU) and the batch-size latency/throughput trade-off.
- Source: LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale: NeurIPS 2022 paper behind bitsandbytes 8-bit inference: vector-wise quantization plus outlier decomposition.
- Source: LM Studio documentation (app and developer docs): LM Studio’s docs: a desktop app running GGUF via llama.cpp and MLX models on Apple silicon, with OpenAI- and Anthropic-compatible local server, lms CLI, SDKs and JIT loading with TTL.
- Source: Long context prompting for Claude 2.1 (Anthropic): Anthropic post showing a needle-in-a-haystack failure was model reluctance, fixed by one prefilled sentence (27% to 98%).
- Source: LongMemEval (Wu et al., 2024): ICLR 2025 benchmark of long-term chat-assistant memory across five abilities, finding about a 30% accuracy drop and proposing indexing, retrieval and reading optimisations.
- Source: Loop Engineering (Loic’s Notebook): The owner’s tutorial on agent loops in Claude Code: triggers, stop criteria, four loop types, and skills as encoded verification standards.
- Source: LoRA Learns Less and Forgets Less: A 2024 Databricks/Columbia study comparing LoRA with full fine-tuning on code and math.
- Source: LoRA Without Regret (Thinking Machines): John Schulman and Thinking Machines Lab’s 2025 study of when LoRA matches full fine-tuning.
- Source: LoRA: Low-Rank Adaptation of Large Language Models: The 2021 Microsoft paper introducing LoRA: frozen weights plus trainable low-rank update matrices.
- Source: Lost in the Middle: How Language Models Use Long Contexts (Liu et al.): Stanford/Berkeley paper showing a U-shaped accuracy curve: models use information at the start and end of long inputs far better than in the middle.
- Source: Lù 2024: BM25S: Technical report and library for a pure-Python BM25 that precomputes scores into sparse matrices at index time, claiming up to 500x speedups over popular Python BM25.
- Source: Ma et al. 2023: Query Rewriting for Retrieval-Augmented LLMs: EMNLP 2023 paper proposing Rewrite-Retrieve-Read, with a small rewriter trained by RL from the reader’s feedback.
- Source: Magpie (Xu et al., 2024): Paper that extracts alignment data from an aligned model by feeding it only the chat template’s user-turn prefix, so it writes both query and answer.
- Source: Making Claude Code more secure and autonomous with sandboxing (Anthropic, 2025): Anthropic engineering post on Claude Code’s OS-level filesystem and network sandbox, which cut permission prompts by 84% internally, plus the cloud git-proxy design.
- Source: Many-shot jailbreaking (Anthropic): Anthropic (Apr 2024): hundreds of faux harmful Q&A turns in a long context jailbreak models; success follows a power law in shot count.
- Source: Markdown exfiltration (Simon Willison tag archive): Willison’s running catalogue of products leaking data through injected Markdown images or links, and why CSP and URL allowlists keep failing.
- Source: Mastering LLM Techniques: Inference Optimization: NVIDIA technical blog surveying prefill/decode, KV cache math, batching, attention variants, parallelism and compression.
- Source: Mastra docs: Evals (scorers) overview: Mastra scorers attached to agents and workflow steps with deterministic trace sampling, stored scores, runEvals, datasets and experiments, and Studio review.
- Source: Mastra docs: Getting started, Agents overview and Agent reference: Mastra’s docs on the Agent class: provider/model router strings, createTool, the Mastra registry, generate vs stream, and the full Agent constructor reference.
- Source: Mastra docs: Memory overview: Mastra memory: message history, working memory, semantic recall and Observational Memory, scoped by resource and thread, backed by a storage adapter.
- Source: Mastra docs: Processors: Mastra input/output/error processors: per-step hooks, tripwire aborts, step retries, and built-ins for moderation, prompt-injection detection, PII redaction and token limits.
- Source: Mastra docs: Supervisor agents: Mastra multi-agent delegation: subagents in the agents map are called as tools, with delegation hooks, message filtering, step-budget caps and memory isolation.
- Source: Mastra docs: Workflows overview: Mastra workflows: createStep/createWorkflow with typed schemas, shared state, nested workflows, run statuses, suspend/resume, and crash recovery via storage leases.
- Source: mastra-ai/mastra repository and licence (GitHub, npm): Mastra’s repository metadata and LICENSE.md: Apache-2.0 for most code, a separate enterprise licence for ee/ directories; npm release cadence.
- Source: Matryoshka Representation Learning (Kusupati et al.): Paper introducing Matryoshka embeddings, trained so that prefixes of a vector are themselves usable lower-dimensional embeddings.
- Source: MCP Authorization Specification (2026-07-28): How a remote MCP server acts as an OAuth 2.1 resource server: protected resource metadata discovery, client registration options, PKCE, resource indicators, issuer validation and step-up scopes.
- Source: MCP Client Registration (spec 2026-07-28): How an MCP client obtains a client_id: pre-registration, Client ID Metadata Documents (preferred for unknown servers) or deprecated Dynamic Client Registration, plus issuer binding.
- Source: MCP Enterprise-Managed Authorization Extension: Official MCP authorization extension that lets an enterprise IdP decide which MCP servers employees can reach, by issuing ID-JAGs that clients trade for MCP access tokens.
- Source: MCP SDKs and Python SDK v2: The official MCP SDK list with tier ratings, plus the Python SDK v2 ‘What’s new’ page (FastMCP renamed MCPServer, 2026-07-28 support, legacy compatibility).
- Source: MCP Security Best Practices: Official catalogue of MCP-specific attacks (confused deputy, token passthrough, SSRF, handle hijacking, local server compromise, malicious auth URLs, mix-up) and their mitigations.
- Source: MCP Security Notification: Tool Poisoning Attacks (Invariant Labs): Invariant Labs (Apr 2025): hidden instructions in MCP tool descriptions can exfiltrate files, shadow other servers’ tools, and change after approval (rug pull).
- Source: MCP Specification 2026-07-28 and Changelog: The current Model Context Protocol revision: a stateless redesign that removes the initialize handshake and sessions, adds server/discover, MRTR and subscriptions/listen, and deprecates Roots, Sampling and Logging.
- Source: MCP Tools Specification (2026-07-28): Spec page for MCP’s tools primitive: listing, calling, content types, structured output, input-required results, stateful handles and the two error channels.
- Source: MCP Transports and Streamable HTTP (2026-07-28): Spec pages defining MCP’s stdio and Streamable HTTP bindings: one POST endpoint, JSON or request-scoped SSE replies, mirrored routing headers, Origin validation and backward-compatibility probing.
- Source: MeanCache, user-centric semantic caching (Gill et al., 2024): A client-side semantic cache that stores conversation context with each query to avoid false hits on follow-ups, and trains its similarity model with federated learning so raw queries never leave the user; IPDPS 2025.
- Source: Measuring AI agent autonomy in practice (Anthropic, 2026): Anthropic research (Feb 2026) on how much oversight real agents get: Claude Code auto-approve and interrupt rates, agent-initiated stops, and human involvement across ~1M API tool calls.
- Source: Measuring Massive Multitask Language Understanding (Hendrycks et al., 2020): The MMLU paper: 57-subject multiple-choice benchmark of world knowledge and problem solving.
- Source: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (Becker et al., 2025): METR randomized controlled trial: experienced OSS developers using early-2025 AI tools (mainly Cursor + Claude 3.5/3.7 Sonnet) took 19% longer, while believing they were faster.
- Source: Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads: Paper adding extra decoding heads plus tree attention as a draft-model-free form of speculative decoding.
- Source: Mem0 documentation, memory concepts: Mem0’s docs on how add extracts facts into SQL, vector and entity stores, and how search ranks memories by semantic, keyword, entity and temporal signals.
- Source: Mem0 documentation, open source overview and Platform vs OSS: Mem0’s docs on deployment modes: OSS defaults (gpt-5-mini, text-embedding-3-small, local Qdrant, SQLite history), the self-hosted server, Platform-only features, and the removal of external graph stores.
- Source: Mem0 pricing page: Mem0 Platform pricing as of November 2026: a free tier metered by add and retrieval requests, and a custom enterprise tier that carries graph memory, Dream, SSO, SLA and on-prem.
- Source: Mem0, the token-efficient memory algorithm (Mem0, 2026): Mem0’s April 2026 redesign: single-pass ADD-only extraction, entity linking and fused semantic/keyword/entity retrieval, with vendor-reported LoCoMo 92.5 and LongMemEval 94.4 at about 7k tokens per query.
- Source: Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory (Chhikara et al., 2025): Paper describing Mem0’s extract-then-update memory pipeline (ADD/UPDATE/DELETE/NOOP) and graph variant, benchmarked on LOCOMO against full context, RAG and other memory systems.
- Source: MemGPT: Towards LLMs as Operating Systems (Packer et al., 2023): Paper that treats the context window like RAM and lets the LLM page information to and from external recall and archival storage through function calls.
- Source: Memory (CrewAI docs): CrewAI’s unified Memory class: LanceDB storage, hierarchical scopes, LLM-assisted save and recall, defaults and limitations.
- Source: Memory tool (Claude API docs): Anthropic’s reference for the client-side memory tool: a /memories file directory Claude can view, create, edit, rename and delete across sessions.
- Source: Merola & Singh 2025: Late Chunking vs Contextual Retrieval: ECIR 2025 workshop paper reproducing late chunking and contextual retrieval: contextual retrieval wins slightly on relevance but costs far more compute; late chunking is model-dependent.
- Source: Messages API reference (Claude API): Reference for Claude’s Create Message endpoint, including max_tokens, stop_sequences, stop_reason values and usage accounting.
- Source: Microsoft Agent Framework overview (Microsoft Learn): Overview of Microsoft Agent Framework, the successor to AutoGen and Semantic Kernel: agents, harness agent, workflows and integrations.
- Source: Microsoft Agent Framework Version 1.0 (Microsoft DevBlogs): Announcement of Microsoft Agent Framework 1.0 GA for .NET and Python, listing stable and preview features.
- Source: Microsoft Foundry — Agent Service Overview: Foundry Agent Service: prompt, voice and hosted agents on the Responses API, toolboxes via MCP, Entra agent identity, VNet isolation and publishing.
- Source: Microsoft Foundry — Azure OpenAI Quotas and Limits: Foundry’s quota model: subscription-level pools, automatic quota tiers (Free + Tiers 1–6), TPM/RPM per model and deployment type, and hard service limits.
- Source: Microsoft Foundry — Configure Network Isolation: Private endpoints, public network access, VNet injection for Agent Service, tool-by-tool private networking support and firewall allowlists.
- Source: Microsoft Foundry — Data, Privacy and Security for Models Sold by Azure: Microsoft’s data-handling commitments for Azure OpenAI and other models sold by Azure: no training use, processing location by deployment type, stored-state features and abuse monitoring.
- Source: Microsoft Foundry — Deployment Types: Foundry’s nine serverless deployment SKUs (Global, Data Zone and regional; Standard, Provisioned and Batch; plus Developer) and their data-processing guarantees.
- Source: Microsoft Foundry — Determine PTU sizing for a workload: Foundry’s PTU sizing method: normalized TPM from peak RPM, prompt/response sizes, output-to-input ratio and cache rate, divided by per-model input TPM per PTU, with minimums, increments and latency targets per model.
- Source: Microsoft Foundry — Foundry Models Overview: Foundry’s model catalogue: models sold by Azure vs partner/community models, serverless vs managed-compute deployment, billing and content-safety differences.
- Source: Microsoft Foundry — Guardrails and Controls Overview: Foundry’s guardrails (formerly content filters): risks, four intervention points including agent tool calls, blocking levels, defaults and override rules.
- Source: Microsoft Foundry — Provisioned Throughput (PTU): How Foundry’s provisioned throughput units work: model-independent quota, separate capacity, sizing, spillover, hourly billing and reservations.
- Source: Microsoft Foundry — Provisioned throughput billing and cost management: How Foundry bills PTUs (hourly on deployed capacity, prorated) versus 1-month/1-year Azure Reservations, why not to scale PTUs with traffic, and how reservation matching and overage work.
- Source: Microsoft Foundry — What Is Microsoft Foundry?: Microsoft’s overview of Foundry (formerly Azure AI Studio / Azure AI Foundry): unified resource model, agents, models, tools and the classic-to-new migration map.
- Source: Microsoft GraphRAG documentation: Docs for Microsoft’s open-source GraphRAG library: indexing pipeline and the global, local, DRIFT and basic query modes.
- Source: Microsoft Research: LazyGraphRAG: Microsoft Research blog on LazyGraphRAG, which defers LLM work to query time and indexes at about vector-RAG cost.
- Source: microsoft/agent-framework README (GitHub): Repository README for Microsoft Agent Framework: packages, feature list (middleware, workflows, OpenTelemetry, declarative YAML, DevUI, Labs, Foundry hosting) and MIT licence.
- Source: microsoft/autogen GitHub README: AutoGen repository README: the project is archived and unmaintained, with Microsoft Agent Framework recommended instead.
- Source: microsoft/semantic-kernel README (GitHub): Semantic Kernel’s README, which now presents Microsoft Agent Framework as its enterprise-ready successor and links the migration guide.
- Source: Mid-conversation system messages (Anthropic): Claude API feature for appending role: system messages mid-conversation, with precedence, cache and placement rules.
- Source: Migrate a Promptfoo eval suite to Langfuse (Langfuse guide): Langfuse’s mapping of promptfoo config concepts onto Langfuse datasets, evaluators and experiments, with the trade-offs of switching.
- Source: Migrate to the Responses API (OpenAI guide): OpenAI’s migration guide from Chat Completions to the Responses API: items vs messages, instructions, storage defaults, state options, tool and schema format changes, and gotchas.
- Source: Milvus Index Explained (Milvus docs): Milvus 3.0 guide to its index types and how to pick one by memory, recall, top-k and filter ratio, with memory estimates.
- Source: MINJA, memory injection attacks on LLM agents (Dong et al., 2025): Paper showing an attacker can poison an agent’s memory bank through ordinary queries alone, with about 98% injection success and 77% attack success across three agents.
- Source: Mistral Large 4 model card (Mistral AI, Hugging Face): Model card for Mistral Large 4: a ~1T-parameter multimodal MoE with ~52B active, 1M context, under a modified-MIT licence restricting MaaS.
- Source: Mistral OCR Announcement: Mistral’s March 2025 launch of an OCR API that returns interleaved Markdown and images from PDFs, with throughput, per-page pricing and internal benchmark claims; since superseded by newer Mistral OCR models.
- Source: Mitigate jailbreaks and prompt injections (Claude docs): Anthropic’s platform guide separating user-as-adversary jailbreaks from indirect injection, with screening, tool_result placement, JSON encoding and least-privilege advice.
- Source: Mitigating the risk of prompt injections in browser use (Anthropic): Anthropic (Nov 2025): RL training, content classifiers and red teaming cut adaptive Best-of-N attack success on its browser agent to about 1-2%, while stating no browser agent is immune.
- Source: MITRE ATLAS data and changelog: MITRE’s ATT&CK-style matrix of adversary tactics and techniques against AI systems; release 2026.10 has 16 tactics, 129 techniques, 44 mitigations and 77 case studies, with a fast-growing agentic section.
- Source: Mixtral of Experts: Mistral AI’s Mixtral 8x7B paper: an 8-expert top-2 sparse MoE (~47B total, ~13B active) that matched Llama 2 70B, plus an analysis of what the router actually learns.
- Source: Mixture of Experts Explained (Hugging Face blog): Hugging Face’s survey of MoE history, load balancing, capacity factor, fine-tuning behaviour and serving trade-offs, written around the Mixtral release.
- Source: MLX and mlx-lm READMEs (Apple ml-explore): Apple’s array framework for Apple silicon (unified memory, lazy evaluation) and its LLM package mlx-lm for generation, quantized conversion, LoRA and serving.
- Source: MMLU-Pro (Wang et al., 2024): Harder, reasoning-heavier MMLU successor with ten answer options and lower prompt sensitivity.
- Source: MMTEB: Massive Multilingual Text Embedding Benchmark (Enevoldsen et al.): Community expansion of MTEB to 500+ tasks and 250+ languages, with cheaper downsampled benchmarks and a refreshed English MTEB(eng, v2).
- Source: Modal Sandboxes guide and networking docs: Modal Sandboxes: gVisor (or VM runtime) containers for untrusted code, 5-minute default lifetime up to 24 hours, filesystem snapshots, and CIDR/SNI-domain egress allowlists.
- Source: Models (OpenAI API docs): OpenAI’s model catalogue: the GPT-6 family (Astra, Sol, Terra, Luna) with context windows, output limits, prices, cutoffs and reasoning-effort levels.
- Source: Models and pricing (DeepSeek API docs): DeepSeek’s API models (V4.1-Flash, V4-Pro): 1M context, thinking modes, cache-hit pricing and off-peak discounts.
- Source: Models and pricing (xAI docs): xAI’s Grok model list with context windows, per-token prices, cached-input prices and the 200K long-context price step.
- Source: Models overview (Mistral docs): Mistral’s model catalogue with open-weight vs premier status, licences, versions and deprecation schedule.
- Source: MTEB: Massive Text Embedding Benchmark (Muennighoff et al.): The paper that defined the standard benchmark for text embedding models and showed no single model wins every task.
- Source: Muse Spark Is Now an Open-Weights Model (PyTorch blog): Meta’s announcement of Muse Spark Open: an 816B/42B-active multimodal MoE with 1M context under the custom Muse Model License.
- Source: NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails: EMNLP 2023 demo paper introducing NeMo Guardrails: a dialogue-manager runtime that enforces Colang rails with three chained LLM calls.
- Source: Nemotron-4 340B Technical Report (NVIDIA, 2024): NVIDIA’s open 340B model family, released for synthetic data generation, whose alignment data was over 98% synthetic: the most detailed public synthetic-data pipeline.
- Source: Neural Machine Translation of Rare Words with Subword Units (Sennrich et al., 2016): The paper that brought byte pair encoding to neural NLP as a way to handle open vocabularies with subword units.
- Source: NeuralTrust, OWASP Top 10 for Agentic Applications 2026: A vendor explainer giving a definition and the main mitigations for each of the ten OWASP agentic risks, ASI01 to ASI10.
- Source: New prompt injection papers: Agents Rule of Two and The Attacker Moves Second (Willison): Simon Willison (Nov 2025) endorses Meta’s Rule of Two as the best practical guidance for agent builders, notes it extends the lethal trifecta to state changes, and challenges its ‘safe’ labels.
- Source: Next-generation Constitutional Classifiers (Anthropic): Anthropic (Jan 2026): an exchange classifier plus a cascade of activation probes cuts jailbreak-defence overhead to about 1% compute and benign refusals to 0.05%.
- Source: NIST AI 600-1, Generative AI Profile: NIST’s cross-sector profile of the AI RMF for generative AI: 12 GAI risks and about 200 suggested actions keyed to Govern/Map/Measure/Manage subcategories.
- Source: NIST AI Risk Management Framework (landing page): NIST’s hub for the voluntary AI RMF 1.0 (January 2023): Govern, Map, Measure, Manage functions, the Playbook, profiles, and the 2026 revision under the US AI Action Plan.
- Source: NoLiMa: Long-Context Evaluation Beyond Literal Matching (ICML 2025): Benchmark where the needle shares almost no words with the question; 11 of 13 128K-class models fall below half their short-context score at 32K.
- Source: Non-determinism in GPT-4 is caused by Sparse MoE (152334H, 2023): Blog post hypothesising that batch-level expert capacity limits in sparse MoE explain GPT-4’s temperature-0 non-determinism, with a small API experiment and explicit caveats.
- Source: Non-Determinism of "Deterministic" LLM Settings (Atil et al., 2024): Empirical study of five LLMs at temperature 0, top-p 1 and fixed seed: accuracy varies up to 15% across runs, best-to-worst gaps reach 70%, and no model gives identical strings consistently.
- Source: Not what you’ve signed up for (indirect prompt injection): Greshake et al. (2023) introduced indirect prompt injection: planting instructions in data an LLM-integrated app will retrieve, demonstrated on Bing Chat.
- Source: Notebook experiment 11: Human-in-the-loop: Runnable LangGraph flow that drafts a refund reply, pauses at an interrupt checkpoint, and resumes on a human approve/reject decision.
- Source: Notebook experiment 13: RAG: Runnable LangChain + FAISS scaffold of the basic split, embed, retrieve, answer pipeline.
- Source: Notebook experiment 14: RAG + CAG: Runnable scaffold that preloads stable company facts (CAG) and retrieves volatile per-order facts (RAG).
- Source: Notebook experiment 15-chain-of-thought (Chain-of-Thought): The owner’s runnable LangChain experiment demonstrating Chain-of-Thought.
- Source: Notebook experiment 16-self-consistency (Self-Consistency): The owner’s runnable LangChain experiment demonstrating Self-Consistency.
- Source: Notebook experiment 17-tree-of-thoughts (Tree of Thoughts): The owner’s runnable LangChain experiment demonstrating Tree of Thoughts.
- Source: Notebook experiment 22: Agentic RAG: Runnable scaffold where retrieval is a tool a ReAct agent calls on demand.
- Source: Notebook experiment 23: Corrective RAG: Runnable scaffold that grades retrieved chunks and falls back to web search when they are weak.
- Source: Notebook experiment 24: Self-RAG: Runnable scaffold of decide-to-retrieve, answer, grounding check, retry.
- Source: Notebook experiment 25: Memory Agent: Runnable scaffold showing thread-scoped short-term memory: a LangGraph ReAct agent with a MemorySaver checkpointer recalls an earlier turn on the same thread_id.
- Source: Notebook tutorial: LangChain vs LlamaIndex: The owner’s comparison of LangChain (orchestration and agents) and LlamaIndex (data ingestion and retrieval), with the same RAG built both ways.
- Source: Notebook, OWASP: The Top 10 Lists and the Security Toolbox: The owner’s tutorial cataloguing OWASP’s Web, API and LLM Top 10 lists and the wider toolbox (ASVS, WSTG, ZAP, SAMM) mapped onto the SDLC.
- Source: Notebook, Web, API & AI Security: A Defensive Guideline: The owner’s tutorial on layered defensive security, including an AI and agent section framed on the OWASP Top 10 for LLM applications.
- Source: Notebook: APIs vs MCP: Owner’s tutorial contrasting plain APIs with MCP: host/client/server roles, the M×N problem, and when an MCP layer is worth it.
- Source: Notebook: Build MCP From Scratch: Owner’s tutorial building an MCP server (FastMCP tools, resource, prompt) and a client that discovers and calls tools, with Inspector and Claude Desktop wiring.
- Source: Notebook: Concurrency design: The owner’s tutorial on concurrency trade-offs (security, idempotency, latency, capacity), safe retries with idempotency keys, and a queue-plus-workers reference architecture for agents.
- Source: Notebook: Driver-Licence Recognizer Lab: The owner’s hands-on lab that trains a YOLO detector to find licence fields, crops them, reads them with EasyOCR, validates with regex and serves JSON via FastAPI, with privacy and production notes.
- Source: Notebook: How LLMs & transformers work: The owner’s interactive tutorial on tokens, embeddings, attention, the transformer block, next-token sampling and training.
- Source: Notebook: LLM Evaluation & Observability: The owner’s tutorial on eval datasets, offline graders, LLM-as-judge, online observability and the eval flywheel.
- Source: Notebook: RAG vs CAG vs GraphRAG: Owner’s tutorial comparing RAG, cache-augmented generation and GraphRAG, with a decision map and a table of RAG variants.
- Source: Notebook: WebMCP & A2A: Owner’s tutorial placing MCP (agent to tools), WebMCP (agent to web page) and A2A (agent to agent) as complementary layers.
- Source: NVIDIA Dynamo (GitHub README): NVIDIA’s open-source datacenter-scale orchestration layer for disaggregated LLM serving over vLLM, SGLang and TensorRT-LLM.
- Source: NVIDIA NeMo Guardrails documentation: NVIDIA’s docs and README for the open-source NeMo Guardrails library: five rail types, config.yml and Colang, self-check rails, streaming and parallel options.
- Source: Ollama documentation (FAQ, context length, Modelfile, OpenAI compatibility, import): Ollama’s official docs on server defaults, VRAM-tiered context length, KV-cache quantization, the Modelfile and its OpenAI-compatible API.
- Source: olmOCR (Poznanski et al., 2025): Ai2’s open toolkit that converts PDFs to linearized reading-order text with a fine-tuned 7B vision-language model, at about $176 per million pages, plus the olmOCR-Bench test set.
- Source: OmniDocBench (Ouyang et al., CVPR 2025) and Leaderboard: A document-parsing benchmark spanning nine document types with 19 layout categories, scoring text, tables, formulas and reading order; its 2026 leaderboard shows specialized document VLMs ahead of general VLMs and pipeline tools.
- Source: On the Brittle Foundations of ReAct Prompting for Agentic LLMs (Verma et al., 2024): Sensitivity study on ALFWorld: ReAct’s gains track exemplar-query similarity, not the interleaving of thoughts or their content.
- Source: On the Theoretical Limitations of Embedding-Based Retrieval (Weller et al.): Google DeepMind paper proving that embedding dimension caps which top-k document sets a single-vector retriever can return, with the LIMIT benchmark to show it.
- Source: On-Policy Distillation (Thinking Machines): Thinking Machines Lab’s 2025 post on distilling by scoring the student’s own samples with a teacher’s per-token log-probs.
- Source: On-Policy Distillation of Language Models (GKD): The 2023 Google DeepMind paper introducing Generalized Knowledge Distillation for autoregressive models.
- Source: Open-source DeepResearch (Hugging Face blog): Hugging Face’s 24-hour open reproduction of OpenAI Deep Research on smolagents: a CodeAgent reached 55.15% on GAIA validation versus about 33% with JSON actions.
- Source: OpenAI acquires Promptfoo to secure its AI agents (TechCrunch, 2026): News report of OpenAI’s March 2026 agreement to acquire Promptfoo and fold it into its Frontier agent platform.
- Source: OpenAI Agents SDK documentation (overview): Overview of the OpenAI Agents SDK: a lightweight Python framework with agents, handoffs/agents-as-tools and guardrails as core primitives.
- Source: OpenAI Agents SDK handoff runtime (turn_resolution.py): openai-agents-python source showing how the Runner resolves handoff tool calls: first handoff wins, extras get an ignore message, server-managed conversations disable filters.
- Source: OpenAI API data residency guide: OpenAI’s regional API domains: ten storage regions, of which only some run inference in-region; per-project or per-request selection and a 10% uplift for newer models.
- Source: OpenAI API pricing, containers row: OpenAI’s price list for hosted containers (Code Interpreter and hosted shell) by memory tier, read 2026-11-14.
- Source: OpenAI Batch API Guide: OpenAI API guide to the Batch API: JSONL input files, 24-hour window, 50% discount, limits, statuses and expiry behaviour.
- Source: OpenAI compatibility (Gemini API docs): Google’s OpenAI-compatible endpoint for Gemini: supported Chat Completions, embeddings, images and batch features, reasoning_effort mapping, extra_body fields and gaps.
- Source: OpenAI Computer Use Guide: OpenAI API guide to computer use in the Responses API: the computer tool loop, code-execution harnesses over Playwright, browser toolsets and safety practices.
- Source: OpenAI Deep Research System Card: OpenAI’s February 2025 system card for deep research: an o3 variant trained with RL on browsing tasks, and its prompt-injection and exfiltration mitigations.
- Source: OpenAI deprecations page (search and deep research models): OpenAI’s deprecations list showing o3/o4-mini deep research and gpt-4o(-mini) search preview models shut down on 2026-07-23.
- Source: OpenAI Deprecations — Fine-tuning Platform Wind-down: OpenAI’s deprecations page entries setting the timeline for ending self-serve fine-tuning.
- Source: OpenAI Function Calling Guide: OpenAI API guide to function calling in the Responses API: strict schemas, tool_choice, parallel calls, custom grammar tools, tool search and streaming.
- Source: OpenAI Image Generation Guide: OpenAI API guide to GPT Image models: Image API vs Responses image_generation tool, edits, masks, markup, sizes, quality, streaming partials, pricing and limitations.
- Source: OpenAI Launches New API, SDK, and Tools to Develop Custom Agents (InfoQ): InfoQ news report on the March 2025 launch of OpenAI’s Responses API, built-in tools and the Agents SDK as Swarm’s successor.
- Source: OpenAI Model Optimization Guide: OpenAI’s guide to fine-tuning methods on its platform, now carrying a wind-down notice.
- Source: OpenAI Model Spec (2026-10-29): OpenAI’s specification of intended model behaviour, including the chain of command that ranks platform, developer, user and untrusted content.
- Source: OpenAI moderation guide: OpenAI’s guide to the free omni-moderation endpoint: 13 categories, per-category scores, image support and inline moderation in the Responses API.
- Source: OpenAI Prompt Caching Guide: OpenAI API guide on automatic prompt caching: prefix routing, prompt_cache_key, retention tiers, TTL pricing on GPT-5.6+, and usage fields.
- Source: OpenAI Realtime API Guide: OpenAI’s guide to the Realtime API for speech-to-speech voice agents: WebRTC vs WebSocket transports, ephemeral client secrets, session config and GA event names.
- Source: OpenAI Realtime Voice Activity Detection Guide: OpenAI’s guide to turn detection in Realtime sessions: server_vad parameters, semantic_vad with eagerness, events and trade-offs.
- Source: OpenAI Reinforcement Fine-Tuning Guide: OpenAI’s documentation for reinforcement fine-tuning (RFT) of o4-mini with graders, plus the API reference for grader types and RFT hyperparameters.
- Source: OpenAI SDK compatibility (Claude docs): Anthropic’s OpenAI-compatible Chat Completions endpoint: what maps, what is ignored or rejected, and why it is meant for testing rather than production.
- Source: OpenAI Speech-to-Text Guide: OpenAI API guide to transcription: current and deprecated STT models, 25 MB file limit, prompting with keywords, diarization, streaming and realtime transcription.
- Source: OpenAI Streaming API Responses Guide: OpenAI API guide on streaming the Responses API with typed semantic SSE events, and the moderation risk of partial output.
- Source: OpenAI Structured Outputs Guide: OpenAI API guide on Structured Outputs vs JSON mode: strict json_schema, supported schema subset, numeric limits, refusals and truncation.
- Source: OpenAI Terms of Use: OpenAI’s consumer/API terms: users own their Output but may not use it to develop models that compete with OpenAI.
- Source: OpenAI Text-to-Speech Guide: OpenAI API guide to the Speech endpoint: deprecated TTS models and Realtime migration, built-in voices, instructions steering, output formats and the AI-voice disclosure rule.
- Source: OpenAI Vector embeddings guide: OpenAI’s API guide to text-embedding-3-small/-large: dimensions, input limits, shortening via the dimensions parameter, normalization and pricing.
- Source: OpenAI Voice Agents Guide: OpenAI’s guide comparing voice-agent architectures (GPT-Live full-duplex, Realtime speech-to-speech, chained STT-agent-TTS) with latency measurement and evaluation advice.
- Source: OpenAI wants to retire SWE-bench Verified (The Decoder, 2026): News report on OpenAI’s February 2026 decision to stop reporting SWE-bench Verified over flawed tasks and contamination.
- Source: OpenAI — Fast mode (formerly priority processing): OpenAI’s pay-per-token premium tier (
service_tier: \"fast\"/\"priority\"): faster, steadier latency, ramp-rate downgrades to Standard, shared rate limits, and its relation to Scale Tier. - Source: OpenAI — Flex processing guide: OpenAI’s
service_tier: \"flex\": Batch-rate pricing on synchronous calls in exchange for slower responses and occasional 429 Resource Unavailable, with longer timeouts and fallback guidance. - Source: OpenAI — Rate limits guide: OpenAI’s rate-limit guide: RPM/RPD/TPM/TPD/IPM per org and project, usage tiers (Free/Build/Launch/Grow), x-ratelimit-* headers, max_tokens reservation, ramp guidance and retry rules.
- Source: OpenAI — Scale Tier and Reserved Tier pages: OpenAI’s committed-capacity offerings: Scale Tier token units (30-day minimum, 99.9% uptime SLA) for models before GPT-5.6, and Enterprise Reserved Tier ($/minute reservations) for GPT-5.6 onward.
- Source: OpenAI, Computer-Using Agent (CUA): OpenAI’s January 2025 announcement of CUA, the GPT-4o-based RL-trained model behind Operator, with OSWorld, WebArena and WebVoyager results.
- Source: openai-agents-python GitHub README: README of the OpenAI Agents SDK repository: core concepts, provider-agnostic claim, licence and requirements.
- Source: openai/evals (GitHub): OpenAI’s open-source eval framework and registry, where basic and model-graded evals are defined in YAML and JSONL.
- Source: OpenLLMetry (Traceloop) README, conventions and ServiceNow announcement: OpenLLMetry, Traceloop’s Apache-2.0 OpenTelemetry instrumentation for LLM providers, vector DBs and agent frameworks: its attribute conventions, content-capture default, and the 2026 ServiceNow acquisition.
- Source: OpenRouter Auto Router docs: Describes openrouter/auto, a hosted per-request model router that rates the request with a fast decision model and picks a model and reasoning effort within a cost tier.
- Source: OpenRouter FAQ (fees, BYOK, free-model limits): OpenRouter’s FAQ: provider prices passed through, a fee on credit purchases, a BYOK allowance, and daily limits on free models.
- Source: OpenRouter provider routing and model fallbacks docs: OpenRouter’s reference for how it load-balances one model across many inference providers, the provider object (order, only, sort, max_price, zdr…), and the models-array fallback.
- Source: OpenSearch blog: Improving document retrieval with sparse semantic encoders: OpenSearch post introducing neural sparse search (2.11) with BEIR relevance, latency and index-size comparisons against BM25, dense and hybrid.
- Source: OpenSearch docs: Normalization processor: OpenSearch search-pipeline processor that normalizes and combines BM25 and vector sub-query scores for hybrid queries.
- Source: OpenTelemetry GenAI conventions: events, tool, agent, MCP and registry details: Detail pages of the semantic-conventions-genai repo: the operation-details and evaluation events, execute_tool and agent spans, MCP spans, message JSON shape and token-count semantics.
- Source: OpenTelemetry semantic conventions for generative AI: The OpenTelemetry GenAI spec: span names, gen_ai.* attributes, token metrics, agent/tool spans and opt-in content capture for LLM telemetry.
- Source: OpenTelemetry semantic-conventions CHANGELOG (GenAI entries): The core semantic-conventions changelog: where the GenAI client conventions started (v1.26.0), the v1.37.0 breaking revamp of message capture and provider naming, and the v1.42.0 move to a dedicated repository.
- Source: Optimizing for cost and intelligence (Anthropic): Anthropic’s measured guidance on multi-model strategies (advisor, orchestrator), effort-based routing and cost per completed task.
- Source: Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs (MIPRO, 2024): The MIPRO paper: jointly optimizing per-module instructions and few-shot demos in LM programs without module-level labels, using grounded proposals and a surrogate model.
- Source: Orca: A Distributed Serving System for Transformer-Based Generative Models: OSDI 2022 paper that introduced iteration-level scheduling, the idea behind continuous batching.
- Source: Orchestrating Agents: Routines and Handoffs (OpenAI Cookbook): OpenAI’s 2024 cookbook that introduced routines and handoffs as transfer functions returning an Agent, the idea behind Swarm and the Agents SDK.
- Source: OSWorld Benchmark: Xie et al. 2024 benchmark of 369 execution-evaluated tasks in real Ubuntu/Windows/macOS environments, plus the OSWorld-Verified refresh and its findings.
- Source: Outlines Documentation and README: dottxt-ai/outlines v1 docs: one call (model(prompt, output_type)) for Literal, regex, CFG and Pydantic outputs; logits-processor masking for local models, schema pass-through for API models; outlines_core, llguidance and xgrammar backends.
- Source: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer: Shazeer et al. (2017) introduce the sparsely-gated MoE layer: a trainable gate picks a few of thousands of feed-forward experts per input, decoupling capacity from compute.
- Source: OWASP GenAI Red Teaming Guide v1.0: OWASP’s January 2025 guide that splits GenAI red teaming into model, implementation, system/infrastructure and runtime (human and agentic) evaluation.
- Source: OWASP LLM02:2025 Sensitive Information Disclosure: OWASP GenAI Security Project entry on how LLM applications leak PII, business data and model details, with mitigations from sanitization to access control and differential privacy.
- Source: OWASP LLM09:2025 Misinformation and LLM10:2025 Unbounded Consumption: The two 2025 OWASP risk pages that absorbed the 2023 entries Overreliance, Model Denial of Service and Model Theft.
- Source: OWASP Top 10 for Agentic Applications (2026): OWASP’s December 2025 list of the ten main risks specific to autonomous agents, from ASI01 Agent Goal Hijack to ASI10 Rogue Agents.
- Source: OWASP Top 10 for LLM Applications 2023-24 (v1.1): The first OWASP LLM Top 10, in its v1.1 form dated April 2024: the list that the 2025 edition reorganised.
- Source: OWASP Top 10 for LLM Applications 2025 (risk pages): The OWASP GenAI Security Project’s 2025 list of the ten most critical LLM application risks, with per-risk definitions, mitigations and attack scenarios.
- Source: OWASP Top 10 for LLM Applications 2026: The August 2026 edition of the OWASP LLM Top 10: Excessive Agency rises to #3, System Prompt Leakage becomes Hidden Context Exposure, and the Agent Control Standard is announced.
- Source: Performant local mixture-of-experts CPU inference with GPU acceleration in llama.cpp: Community guide on Hugging Face to hybrid CPU+GPU MoE inference in llama.cpp: keep attention and shared experts on GPU, routed experts in system RAM.
- Source: Perplexity API docs (Search API, Agent API models and effort levels): Perplexity’s developer docs as of late 2026: the raw Search API priced per request, the Sonar grounded model, and Agent API effort presets that bundle a model, step budget and web tools.
- Source: Pervasive Annotation Errors Break Text-to-SQL Benchmarks (Jin et al., 2026): Expert audit finding 52.8% of BIRD Mini-Dev and 62.8% of audited Spider 2.0-Snow examples have annotation errors, enough to shift BIRD agent rankings by up to 9 places.
- Source: pgvector README: Official README of pgvector, the PostgreSQL extension for vector similarity search: types, operators, HNSW and IVFFlat indexes, iterative scans and filtering.
- Source: Pinecone serverless architecture (Pinecone blog): Pinecone’s description of its serverless design: blob storage as source of truth, geometric partitioning, a freshness layer, namespaces and cached compute.
- Source: Playwright MCP README: Microsoft’s Playwright MCP server: browser automation for LLMs via accessibility-tree snapshots rather than screenshots, with opt-in vision mode and security caveats.
- Source: Plugin manifest reference (Claude Code docs): The plugin.json schema and standard layout for Claude Code plugins, which bundle skills, commands, agents, hooks, MCP and LSP servers into one namespaced, versioned unit.
- Source: Portkey AI Gateway docs (fallbacks, retries, simple and semantic cache): Portkey’s gateway config reference: ordered fallback targets on status codes, retries with exponential backoff or Retry-After, and exact or semantic caching with documented limits.
- Source: Presidio documentation: Open-source PII detection and de-identification SDK: analyzer (NER + regex + checksums + context), anonymizer operators, image and structured-data redaction, with an explicit no-guarantee warning.
- Source: Press et al. 2022: Self-Ask (Measuring and Narrowing the Compositionality Gap): Findings of EMNLP 2023 paper showing models answer sub-questions they cannot compose, and that asking explicit follow-up questions (optionally answered by search) narrows the gap.
- Source: Pricing (Anthropic Claude API): Anthropic’s full price list: per-model token prices, cache multipliers, batch discount, long-context rules, data-residency and fast-mode premiums.
- Source: Pricing (OpenAI API): OpenAI’s per-token prices across Standard, Batch/Flex and Priority tiers, including cached input and long-context surcharges.
- Source: Processes (CrewAI docs): CrewAI’s sequential and hierarchical processes and how tasks are assigned and validated in each.
- Source: Prompt & Context Engineering (Notebook tutorial): The owner’s tutorial on prompt anatomy, prompting techniques, structured output, the context window, prompt caching and context engineering.
- Source: Prompt design strategies (Google Gemini): Google’s Gemini prompting guide: instructions, constraints, few-shot formatting, prefixes, decomposition, sampling parameters, system instructions and Gemini 3 advice.
- Source: Prompt engineering guide (OpenAI): OpenAI’s API guide to prompting: message roles, the instructions parameter, Markdown/XML structure, few-shot, prompts-in-code versioning and snapshot pinning.
- Source: Prompt engineering overview (Anthropic): Anthropic’s entry page on when prompt engineering is the right tool and what to have in place before starting.
- Source: Prompt injection and jailbreaking are not the same thing: Simon Willison (Mar 2024): injection attacks applications by concatenating untrusted input with trusted prompts; jailbreaking attacks the model’s own safety training.
- Source: Promptfoo docs: Assertions and metrics: Promptfoo reference for its YAML test config and deterministic, model-graded, trace and tool-call assertion types.
- Source: Promptfoo docs: CLI, configuration guide, CI/CD and GitHub Action: Promptfoo reference for promptfooconfig.yaml structure, eval CLI flags, exit codes and pass-rate thresholds, caching, and running evals and red teams in CI.
- Source: Promptfoo docs: Red team overview and configuration: How promptfoo red teaming composes plugins (what to attack) with strategies (how to deliver it), plus framework presets, targets and remote generation.
- Source: Promptfoo docs: Red team plugins and quickstart: Promptfoo’s catalogue of 157 red-team plugins (PII, BOLA/BFLA, RBAC, SSRF, injection, MCP, cross-session leak) and the setup/run/report quickstart flow.
- Source: Prompting best practices (Anthropic): Anthropic’s living reference for prompting current Claude models: clarity, examples, XML, roles, long context, prefill migration, thinking and chaining.
- Source: Prompting Science Report 2: The Decreasing Value of Chain of Thought: Meincke, Mollick, Mollick and Shapiro (2025): explicit CoT prompts give small, variable gains on non-reasoning models and only marginal gains on reasoning models at much higher time and token cost.
- Source: Pydantic AI documentation: Official Pydantic AI docs: the typed Agent, dependency injection via RunContext, output modes, tools, usage limits, capabilities, durable execution, testing and the V2 changelog.
- Source: Pydantic AI framework comparison pages (vs Vercel AI SDK, vs LangChain/LangGraph): Vendor-written comparison tables positioning Pydantic AI against the Vercel AI SDK and LangChain/LangGraph, including how to put a Python agent behind a useChat frontend.
- Source: PyRIT (Microsoft) repository, docs and releases: Microsoft AI Red Team’s open-source framework for composing targets, converters, attacks and scorers into automated, multi-turn red-team scenarios; v1.2.0 released 2026-11-13.
- Source: Qdrant docs: Vectors (multivectors): Qdrant’s vector concepts page: native multivector points scored with MaxSim, configuration, and the hard per-point size limit.
- Source: Qdrant indexing documentation: Qdrant’s reference for payload indexes, HNSW parameters, filterable HNSW, the query planner, ACORN, sparse vector indexes and tenant-aware indexing.
- Source: Qdrant tutorial: PDF Retrieval at Scale (ColPali/ColQwen): Qdrant tutorial that makes ColPali/ColQwen indexing practical by HNSW-indexing row/column mean-pooled multivectors and reranking with the full multivector, which is stored without an HNSW graph.
- Source: QLoRA: Efficient Finetuning of Quantized LLMs: The 2023 paper that backpropagates through a frozen 4-bit base model into LoRA adapters.
- Source: Qualys, What’s changed in the OWASP Top 10 for LLMs 2025: A vendor walkthrough of how the 2025 OWASP LLM list differs from the 2023 list, with the stated reason for each new or reworked entry.
- Source: Quantization & Pruning: Make Models Small and Fast (Notebook tutorial): The owner’s tutorial on quantization (PTQ/QAT, INT8/INT4, GPTQ/AWQ/GGUF) and pruning.
- Source: Qwen2.5-72B-Instruct config.json: Model configuration file used as a worked example for KV cache sizing.
- Source: Qwen3 Technical Report and launch blog: Qwen3’s MoE models (235B-A22B, 30B-A3B): 128 fine-grained experts with 8 active, no shared experts, and a global-batch load-balancing loss.
- Source: Qwen3.8 model cards (Qwen, Hugging Face): Model cards for Qwen3.8-27B (dense, Apache 2.0) and Qwen3.8-2.4T-A95B (MoE, custom Qwen3.8-Max licence).
- Source: Rackauckas 2024: RAG-Fusion: Case study (Infineon) of generating several query variants, retrieving for each and merging with reciprocal rank fusion; better coverage, some off-topic drift.
- Source: Ragas docs: Answer Relevancy and Noise Sensitivity metrics: Ragas documentation of the embedding-based Answer Relevancy metric and the Noise Sensitivity error-rate metric.
- Source: Ragas docs: Available metrics: Documentation of the open-source Ragas library’s RAG metrics (context precision/recall, faithfulness, response relevancy, noise sensitivity).
- Source: Ragas docs: context precision, context recall, faithfulness, and release notes: Per-metric Ragas documentation (formulas, collections vs legacy API, non-LLM and ID-based variants) plus the 0.3-0.4 release history.
- Source: Ragas docs: Testset generation for RAG: Ragas guide to generating synthetic RAG test questions from your own documents via a knowledge graph and query synthesizers.
- Source: Raising the bar on SWE-bench Verified with Claude 3.5 Sonnet (Anthropic, 2025): Anthropic’s description of a minimal bash-plus-edit-tool scaffold that reached 49% on SWE-bench Verified, with lessons on tool design.
- Source: ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2022): Yao et al. (ICLR 2023): prompt an LLM to interleave free-text thoughts with environment actions and observations; beats act-only and imitation/RL baselines on ALFWorld and WebShop.
- Source: Rearchitecting Letta’s Agent Loop (Letta, 2025): Letta’s letta_v1_agent drops MemGPT heartbeats, the send_message tool and tool-argument inner thoughts in favour of native reasoning and native multi-step tool use.
- Source: Reasoning best practices (OpenAI): OpenAI’s guide to when to use reasoning models and how prompting them differs: simpler prompts, no CoT instructions, zero-shot first.
- Source: Reasoning models don’t always say what they think (Anthropic): Anthropic Alignment Science (2025): reasoning models usually fail to mention hints they used, limiting chain-of-thought monitoring.
- Source: Reasoning models guide (OpenAI API docs): OpenAI’s reference for reasoning models: reasoning.effort, hidden reasoning tokens, summaries, encrypted reasoning items, context preservation and token limits.
- Source: Red Teaming Language Models to Reduce Harms: Ganguli et al. (Anthropic, Aug 2022): 38,961 human red-team attacks across 2.7B-52B models; RLHF models get harder to red team as they scale.
- Source: Red Teaming Language Models with Language Models: Perez et al. (DeepMind, Feb 2022): use one LM to generate test cases and a classifier to grade a target LM, from zero-shot prompting to RL-trained red LMs.
- Source: Redis LangCache docs (overview and API/SDK examples): Redis’s managed semantic-caching service: a search-then-store REST API with automatic embeddings, similarity thresholds, attribute scoping, exact/semantic search strategies and attribute-based deletion; in preview as of late 2026.
- Source: RedisVL SemanticCache user guide: The open-source RedisVL library’s SemanticCache: cosine-distance thresholds, TTL with refresh-on-hit, tag/numeric filters for per-user scoping, and drop/clear APIs.
- Source: Reduce hallucinations (Claude Docs): Anthropic’s guide to prompting techniques that reduce hallucinations: allow uncertainty, ground in quotes, cite and verify.
- Source: Reducing latency (Anthropic docs): Anthropic’s latency guidance: measure baseline latency and TTFT, pick a faster model and lower effort, trim prompt and output tokens, and stream.
- Source: Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023): NeurIPS 2023: an agent retries a task, writing verbal self-reflections on each failure into an episodic memory instead of updating weights.
- Source: Restate docs: durable execution and the human-in-the-loop agent pattern: Restate’s journal-based durable execution server (single Rust binary, virtual objects, idempotency keys, suspension on FaaS) and its awakeable-based approval pattern for agents.
- Source: Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?: Min et al. (2022): in few-shot classification, label correctness matters little; label space, input distribution and format drive gains.
- Source: ReWOO: Decoupling Reasoning from Observations (Xu et al., 2023): Plan all tool calls up front with evidence variables, execute, then solve: about 5x fewer tokens than ReAct on HotpotQA with slightly higher accuracy.
- Source: RFC 8693 OAuth 2.0 Token Exchange: IETF standard (Jan 2020) for trading one security token for another at an STS, with explicit impersonation vs delegation semantics and the act / may_act claims.
- Source: RLAIF vs. RLHF: Google’s 2023 study finding that AI-labelled preferences can match human preferences for RL fine-tuning.
- Source: Robertson & Zaragoza 2009: The Probabilistic Relevance Framework: BM25 and Beyond: The canonical survey deriving BM25 and BM25F from the probabilistic relevance framework, with guidance on k1 and b.
- Source: Robust Speech Recognition via Large-Scale Weak Supervision (Whisper): Radford et al. 2022 paper introducing Whisper, a speech recognizer trained on ~680k hours of weakly supervised multilingual audio that generalizes zero-shot.
- Source: RoFormer: Rotary Position Embedding (Su et al., 2021): Introduces RoPE, which encodes position by rotating query and key vectors so attention scores depend on relative distance.
- Source: Role-based access control for Azure OpenAI: The built-in Azure RBAC roles for Azure OpenAI (OpenAI User, OpenAI Contributor, Cognitive Services Contributor, Usages Reader) and what each can and cannot do.
- Source: RouteLLM (Ong et al. 2024): Paper training routers on human preference data to send each query to a strong or weak LLM, cutting cost over 2x without quality loss.
- Source: RouteLLM, an open-source framework for cost-effective LLM routing (LMSYS blog): LMSYS write-up of RouteLLM: four router architectures trained on Chatbot Arena preferences, with cost savings measured on MT Bench, MMLU and GSM8K for GPT-4 Turbo vs Mixtral 8x7B.
- Source: RouterBench, a benchmark for multi-LLM routing (Hu et al., 2024): Benchmark of 405k inference outcomes across 11 models showing a large oracle-routing headroom, that simple predictive routers rarely beat a cost-quality baseline, and that cascades collapse once the judge errs more than about 20% of the time.
- Source: RULER: What’s the Real Context Size of Your Long-Context Language Models? (NVIDIA): NVIDIA benchmark extending needle-in-a-haystack with multi-needle, multi-hop and aggregation tasks; about half of 32K+ models fail at 32K.
- Source: Running agents (OpenAI Agents SDK docs): The OpenAI Agents SDK Runner loop, max_turns, RunConfig, conversation-state options, error handlers, exceptions and resumable RunState.
- Source: s1 — Simple Test-Time Scaling (Muennighoff et al., 2025): Paper showing 1,000 curated reasoning traces plus ‘budget forcing’ give a 32B model controllable test-time scaling that beats o1-preview on competition math.
- Source: safetensors docs/README and Hub pickle-scanning docs: Why pickle checkpoints can execute arbitrary code, how the Hub scans them, and the safetensors format (8-byte length, JSON header, raw buffer) that replaces them.
- Source: Same fabricated answer, RAGAS scored 0.0 and DeepEval 1.0 (Nick Lamb): Small practitioner experiment showing RAGAS and DeepEval ‘faithfulness’ measure different things, plus judge variance and cost figures.
- Source: Santhanam et al. 2021: ColBERTv2: NAACL 2022 paper adding denoised distillation and centroid-plus-residual compression to ColBERT, shrinking the MS MARCO index from 154 GiB to 16-25 GiB with no quality loss at 2 bits.
- Source: Santhanam et al. 2022: PLAID: Paper describing PLAID, the ColBERTv2 search engine that prunes candidates using centroids, cutting latency up to 7x on GPU and 45x on CPU.
- Source: Sarathi-Serve: Taming Throughput-Latency Tradeoff in LLM Inference: OSDI 2024 paper introducing chunked prefills and stall-free scheduling.
- Source: Say What You Mean (.txt response to Let Me Speak Freely): Will Kurt (.txt): re-runs the ‘Let Me Speak Freely’ experiments and finds structured generation matches or beats unstructured output when prompts are fair.
- Source: Scaling Data-Constrained Language Models (Muennighoff et al., 2023): 400-run study showing up to about 4 epochs of repeated data is almost as good as fresh data, with value decaying after that; proposes a scaling law for repeated tokens.
- Source: Scaling Laws for Neural Language Models (Kaplan et al., 2020): OpenAI’s 2020 paper showing language-model loss follows smooth power laws in parameters, data and compute, and recommending big models trained short of convergence.
- Source: Scaling LLM Test-Time Compute Optimally (Snell et al., 2024): Paper showing that difficulty-adaptive test-time compute beats best-of-N by ~4x and can beat a 14x larger model on easy and medium problems.
- Source: Security Affairs: CISA adds LiteLLM CVE-2026-42208 to KEV: News report that CISA listed the LiteLLM proxy SQL injection as known-exploited in May 2026, with a May 11 federal deadline.
- Source: Security Affairs: CISA adds LiteLLM CVE-2026-42271 to KEV: News report that CISA listed LiteLLM’s MCP test-endpoint command injection as known-exploited in June 2026.
- Source: Self-Consistency Improves Chain of Thought Reasoning in Language Models: Wang et al. (2022): sample many chain-of-thought paths and take a majority vote over final answers instead of greedy decoding.
- Source: Self-Instruct (Wang et al., 2022): The paper that bootstrapped 52K instructions from 175 human seed tasks using GPT-3 itself, establishing the template for LLM-generated instruction data.
- Source: Semantic Kernel to Microsoft Agent Framework Migration Guide (Microsoft Learn): Official mapping from Semantic Kernel agents (Kernel, plugins, threads, invoke) to Agent Framework (AIAgent/Agent, direct tools, sessions, run) in .NET and Python.
- Source: Sentence-Transformers docs: Cross-Encoders: SBERT documentation contrasting bi-encoders with cross-encoders and recommending retrieve-then-rerank.
- Source: SentencePiece (Kudo & Richardson, 2018): A language-independent tokenizer library that trains BPE or Unigram models directly on raw text, treating whitespace as a symbol.
- Source: SerpApi Google Search API docs: SerpApi’s reference for scraping Google results pages into structured JSON: parameters, localisation, one-hour cache and result blocks.
- Source: Server tools (Claude docs): Anthropic’s shared mechanics for server-executed tools: server_tool_use blocks, the server-side loop and pause_turn, mixing server and client tools, allowed_callers and domain filtering.
- Source: Sessions (OpenAI Agents SDK docs): How OpenAI Agents SDK sessions store and replay conversation history, their backends, and their security and compaction caveats.
- Source: SGLang documentation (home): Official SGLang docs landing page: positioning, hardware support and adoption claims as of late 2026.
- Source: SGLang: Efficient Execution of Structured Language Model Programs: Paper introducing SGLang, RadixAttention and compressed-FSM constrained decoding.
- Source: Shao et al. 2024: STORM: Stanford paper on writing Wikipedia-like articles from scratch by discovering perspectives, simulating grounded expert conversations, and outlining before writing.
- Source: Shell tool guide (OpenAI docs): OpenAI’s shell tool in hosted (managed Debian container, network off by default, org allowlist plus per-request network_policy, sidecar-injected domain secrets) and local modes.
- Source: Singh et al. 2025: Agentic Retrieval-Augmented Generation, a Survey: Survey that classifies agentic RAG architectures (router, multi-agent, hierarchical, corrective, adaptive, graph-based, document workflows).
- Source: Skill authoring best practices (Claude Platform docs): Anthropic’s guide to writing skills: conciseness, degrees of freedom, third-person trigger descriptions, one-level references, checklists, validator loops, utility scripts and eval-first development.
- Source: SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks (Li et al., 2026): Benchmark of curated Agent Skills on verifiable tasks: about +16.6 pp average pass rate, focused skills beat exhaustive ones, and self-written skills fell below the no-skill baseline.
- Source: Sleep-time Compute (Letta blog, 2025): Letta 0.7.0 announcement of sleep-time agents: a background agent owns memory editing and rewrites the primary agent’s memory blocks asynchronously.
- Source: Sleep-time Compute: Beyond Inference Scaling at Test-time (Lin et al., 2025): Paper proposing that models pre-process a context offline, before queries arrive, cutting test-time compute about 5x at equal accuracy when queries are predictable.
- Source: smolagents API reference: Agents (v1.26.0): Constructor parameters and defaults for MultiStepAgent, CodeAgent and ToolCallingAgent, run() options, AgentMemory, prompt templates and save/push_to_hub.
- Source: smolagents conceptual guides: What are agents? and How do multi-step agents work?: Hugging Face’s agency spectrum (processor to code agent), when to avoid agents, the case for code actions, and the MultiStepAgent ReAct loop with memory steps.
- Source: smolagents docs: Index and Guided tour (Hugging Face): smolagents overview and guided tour: CodeAgent vs ToolCallingAgent, model classes, tools, authorized imports, executors, final_answer_checks, managed agents, Gradio UI and Hub sharing.
- Source: smolagents docs: Inspecting runs with OpenTelemetry: smolagents tracing via the OpenInference SmolagentsInstrumentor (OpenTelemetry) into Phoenix or Langfuse, plus MLflow’s own one-line autologging.
- Source: smolagents docs: Manage your agent’s memory: How to inspect and rewrite smolagents agent memory: replay(), memory.steps, step_callbacks that prune old screenshots, and driving an agent one step at a time with agent.step().
- Source: smolagents docs: Secure code execution: Threat model for code agents, the AST-walking LocalPythonExecutor and its limits, and the two sandboxing approaches (snippet-in-sandbox vs whole-agent-in-sandbox) with E2B, Docker, Modal and Blaxel.
- Source: smolagents docs: Tools (advanced usage): smolagents tool tutorial: Tool subclasses, Hub sharing with trust_remote_code, MCPClient and ToolCollection for MCP servers, structured_output for MCP outputSchema, Gradio Spaces and LangChain tools.
- Source: smolagents on PyPI: PyPI listing for smolagents: version 1.26.0 released 2026-05-29, Python 3.10+, and optional extras for each model backend, executor and integration.
- Source: SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models: ICML 2023 paper enabling W8A8 by migrating activation outliers into the weights.
- Source: Snowflake Cortex Analyst Verified Query Repository Docs: Snowflake documentation for the Verified Query Repository: curated question-SQL pairs stored in a semantic model that Cortex Analyst reuses when a new question is similar.
- Source: So You Want to Hire a Forward Deployed Engineer (First Round Review): First Round Review’s 2026 guide on when a startup needs FDEs, what they do, how they fail and how to hire them, from ex-Palantir, Ironclad, Looker and Serval leaders.
- Source: SOC 2 Type 2 offering (Microsoft Compliance): Microsoft’s explainer of SOC 2 attestations: AICPA standards, Trust Services Criteria, Type 1 versus Type 2, the 12-month period of performance, bridge letters and user entity responsibilities.
- Source: SOC FAQs (AWS): AWS’s description of SOC 1, SOC 2 and SOC 3 reports, their audiences, NDA access via AWS Artifact, reporting cadence and bridge letters.
- Source: Spider (Yu et al., EMNLP 2018): The cross-domain text-to-SQL benchmark of 10,181 questions over 200 multi-table databases, whose split by database forces models to generalize to unseen schemas.
- Source: Spider 2.0 (Lei et al., ICLR 2025) and Leaderboard: Enterprise text-to-SQL benchmark of 632 workflow tasks on BigQuery, Snowflake and dbt projects with 1,000+ column schemas; o1-preview agents solved 21.3% at release, and agentic systems reached roughly 74-97% by late 2026.
- Source: SR 26-2, Revised Guidance on Model Risk Management (Federal Reserve): The April 2026 interagency letter that rescinds SR 11-7 and replaces it with risk-based model risk management guidance, aimed mainly at banks over $30B in assets.
- Source: STaR — Self-Taught Reasoner (Zelikman et al., 2022): Stanford/Google paper that bootstraps reasoning by fine-tuning a model on its own rationales that reach correct answers: the template for rejection-sampling fine-tuning.
- Source: State (ADK sessions docs): ADK session state: key prefixes and scopes, safe ways to update state, instruction templating and SessionService backends.
- Source: Subagents in the SDK (Claude Agent SDK docs): Defining, invoking, resuming and capping subagents in the Claude Agent SDK via AgentDefinition and the Agent tool.
- Source: Sun et al. 2023: RankGPT (Is ChatGPT Good at Search?): EMNLP 2023 paper showing instructed LLMs can rerank as well as supervised rerankers, and distilling that into small models.
- Source: SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering (Yang et al., 2024): Paper introducing the agent-computer interface (ACI) idea: purpose-built search, windowed viewing and lint-guarded editing commands that markedly improve coding agents.
- Source: SWE-bench evaluation harness documentation: Official guide to the Docker-based SWE-bench harness that applies model patches and runs FAIL_TO_PASS and PASS_TO_PASS tests.
- Source: SWE-Bench Pro (Deng et al., Scale AI, 2025): Scale AI’s contamination-resistant, long-horizon successor benchmark: 1,865 tasks from 41 repos with public, held-out and commercial subsets; best public score rose from 23.3% (v1) to 43.6% (v2).
- Source: SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (Jimenez et al., 2023): The SWE-bench paper: 2,294 real GitHub issues from 12 Python repos, graded by running the repo’s tests.
- Source: Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity: Fedus, Zoph and Shazeer simplify MoE to top-1 routing and define the capacity factor, auxiliary balance loss, selective router precision and expert dropout used by later MoEs.
- Source: TableFormer (Nassar et al., 2022) and OTSL (Lysak et al., 2023): IBM Research papers on table structure recognition: TableFormer predicts table structure plus cell boxes with transformer decoders; OTSL is a 5-token table language that halves sequence length versus HTML.
- Source: Tantivy bm25.rs (source code): Tantivy’s BM25 scorer: hard-coded k1 = 1.2 and b = 0.75, Lucene-style IDF, (k1+1) factor, and a 256-entry fieldnorm cache.
- Source: Tasks (CrewAI docs): CrewAI Task attributes, context passing between tasks, structured outputs and function or LLM-based task guardrails.
- Source: Tavily API docs (Search endpoint and credits): Tavily’s reference for its LLM-oriented search API: search depths, chunked results, raw content, domain/time filters and credit pricing for search, extract, map, crawl and research.
- Source: Temporal docs: Workflow definitions, Event History, Activities and retry policies: Official Temporal docs on deterministic Workflow replay against an append-only Event History, history limits, Continue-As-New, Activities, idempotency and default retry policies.
- Source: Temporal OpenAI Agents SDK integration (Python docs): Temporal’s plugin that runs OpenAI Agents SDK agents as Workflows with each model call as an Activity, plus activity-backed tools, MCP, HITL via signals, and its limitations.
- Source: TensorRT-LLM overview: NVIDIA’s official overview of TensorRT-LLM, its PyTorch-based LLM inference library.
- Source: Terminal-Bench (Merrill et al., 2026): Paper introducing Terminal-Bench 2.0: 89 hard, realistic command-line tasks with per-task environments and verification tests.
- Source: Text Embeddings by Weakly-Supervised Contrastive Pre-training (E5): Microsoft paper introducing the E5 open embedding models, trained contrastively on a large curated set of weakly supervised text pairs.
- Source: Text Embeddings Reveal (Almost) As Much As Text: Morris et al. (EMNLP 2023): Vec2Text iteratively inverts dense embeddings, recovering 92% of 32-token inputs exactly and full names from clinical notes.
- Source: Text Generation Inference (TGI) documentation: Hugging Face’s TGI serving toolkit docs, which now state the project is in maintenance mode.
- Source: Textbooks Are All You Need (phi-1): Microsoft Research paper training a 1.3B code model on filtered ’textbook-quality’ web data plus GPT-3.5-generated synthetic textbooks and exercises.
- Source: The "think" tool (Anthropic Engineering, 2025): Anthropic engineering post on a no-op ’think’ tool for mid-task reasoning, compared with extended thinking; later superseded by interleaved thinking.
- Source: The Anatomy of a Modern AI Agent: Blog post mapping an AI agent as a loop plus tools, context, delegation, verification and guardrails.
- Source: The Anatomy of an Agent Harness (LangChain, 2026): LangChain post defining the harness as everything that isn’t the model and walking through the components that give a model durable state, execution and long-horizon behaviour.
- Source: The Art of Scaling Reinforcement Learning Compute for LLMs (Khatri et al., 2025): Meta-led 400,000 GPU-hour study fitting sigmoid compute-performance curves to LLM RL and proposing the ScaleRL recipe.
- Source: The Attacker Moves Second: Nasr, Carlini, Tramèr et al. (Oct 2025): adaptive attacks exceed 90% success against most of 12 published jailbreak and prompt-injection defences that reported near-zero ASR.
- Source: The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management (Lindenbauer et al., 2025): JetBrains Research study on SWE-bench Verified showing that masking old tool observations halves agent cost and matches LLM summarisation, which also lengthens trajectories.
- Source: The Curious Case of Neural Text Degeneration (Holtzman et al., 2020): Introduces nucleus (top-p) sampling and shows that likelihood-maximising decoding produces bland, repetitive text.
- Source: The Curse of Recursion / Model Collapse (Shumailov et al.): Paper showing that training generative models recursively on model-generated data erases the tails of the original distribution (‘model collapse’).
- Source: The Digital Omnibus on AI enters into force (Lewis Silkin): Law-firm briefing on Regulation (EU) 2026/1744, which amends the AI Act: high-risk deadlines moved to December 2027 and August 2028, softer AI literacy duty, marking grace period, new prohibitions.
- Source: The Dual LLM pattern for building AI assistants that can resist prompt injection: Simon Willison (2023): split an assistant into a tool-wielding privileged LLM that never sees untrusted text and a tool-less quarantined LLM, coordinated by plain code.
- Source: The early bird catches the leak, timing side channels in LLM serving (Song et al., 2024): Shows that shared KV caches and semantic caches leak other users’ prompts through hit/miss timing (KV-cache attack on SGLang, semantic-cache attack on GPTCache in a LangChain app); accepted to IEEE TIFS.
- Source: The Faiss Library (Douze et al.): Paper describing Faiss’s design space for vector search: IVF, product and additive quantization, refinement, graph indexes, filtering and index selection.
- Source: The False Promise of Imitating Proprietary LLMs: Berkeley 2023 paper finding that fine-tuning open models on ChatGPT outputs copies style, not capability or factuality.
- Source: The FineWeb Datasets (Penedo et al., Hugging Face, 2024): Hugging Face’s 15T-token open web pretraining dataset from 96 Common Crawl snapshots, with ablated filtering and dedup choices and the 1.3T-token FineWeb-Edu subset.
- Source: The GenAI Divide: State of AI in Business 2025 (MIT NANDA): MIT NANDA’s 2025 preliminary report finding that about 5% of enterprise GenAI pilots produce measurable P&L impact, blaming a ’learning gap’ in integration rather than model quality.
- Source: The illusion of a perfect metric (Oliva et al., 2025): Survey arguing no automatic text metric, including LLM-as-judge and RAG metrics, is reliably validated against human judgment.
- Source: The Instruction Hierarchy (OpenAI): Wallace et al. (2024): train models to prioritise privileged (system/developer) instructions over lower-privileged text to resist injection.
- Source: The Leaderboard Illusion (Singh et al., 2025): Paper documenting private variant testing, selective disclosure and data asymmetries that distort Chatbot Arena rankings.
- Source: The lethal trifecta for AI agents: Simon Willison (June 2025): an agent with private data, exposure to untrusted content and an external communication channel can be made to exfiltrate data; remove one leg.
- Source: The Llama 3 Herd of Models (Meta, 2024): Meta’s Llama 3 paper; its post-training sections document large-scale rejection sampling, LLM-judge filtering and execution-verified synthetic data.
- Source: The Llama 4 herd (Meta AI blog): Meta’s Llama 4 launch: Scout (109B/17B active, 16 experts) and Maverick (400B/17B active, 128 routed + 1 shared), Meta’s first MoE models, with alternating dense and MoE layers.
- Source: The MCP Registry (official docs): Docs for the official MCP Registry: a metadata-only catalogue of public MCP servers with namespace verification, meant to feed downstream aggregators.
- Source: The Open Source AI Definition 1.0 (OSI): OSI’s definition of open-source AI: four freedoms plus data information, code and parameters; weights alone don’t qualify.
- Source: The Palantirization of Everything (a16z): Marc Andrusko’s 2026 critique of startups copying Palantir’s forward-deployed model: when it pays off, the services trap, and how to treat deployment as scaffolding.
- Source: The Rule of 2 (Chromium security docs): Chromium’s security policy: code may combine at most two of untrustworthy input, a memory-unsafe language, and high privilege; the ancestor of Meta’s Agents Rule of Two.
- Source: The TREC-8 Question Answering Track Report (Voorhees): Report on the first large-scale open-domain QA evaluation, whose ranked-answer scoring popularised mean reciprocal rank (MRR).
- Source: tiktoken (OpenAI): OpenAI’s open-source byte-level BPE tokenizer library and the encodings (cl100k_base, o200k_base) used by its models.
- Source: Timeouts, retries, and backoff with jitter (Amazon Builders’ Library): Marc Brooker’s Builders’ Library essay: set timeouts from downstream latency percentiles, retry at one layer only, limit retries with a token bucket, back off with jitter, and retry only idempotent calls.
- Source: Token counting (Claude API docs): Anthropic’s documentation for the free count_tokens endpoint, including the tokenizer change from Claude Opus 4.7 onward.
- Source: Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs: Paper showing GPT-3.5/GPT-4 arithmetic depends on whether digits are chunked left-to-right or right-to-left.
- Source: Tools, agents as tools (OpenAI Agents SDK docs): The OpenAI Agents SDK tools page, read for its agents-as-tools section: Agent.as_tool() options, nested input, state isolation and output extraction.
- Source: Towards Deterministic Inference in SGLang and Reproducible RL Training (LMSYS): SGLang team’s post adding a deterministic-inference mode built on batch-invariant kernels, with seeded sampling, CUDA-graph support, overhead benchmarks and an RL reproducibility recipe.
- Source: Towards Understanding Sycophancy in Language Models (Anthropic): Anthropic paper showing five production assistants are sycophantic and tracing it partly to human preference data and preference models.
- Source: Tracing (OpenAI Agents SDK docs): Built-in tracing in the OpenAI Agents SDK: default spans, disabling, ZDR caveat, custom processors and sensitive-data controls.
- Source: Trading Margin for Moat / Services-Led Growth (a16z): Joe Schmidt’s 2025 argument that complex enterprise AI apps should accept services-heavy implementation (FDEs) early, as Salesforce, ServiceNow and Workday did in the cloud shift.
- Source: Training Compute-Optimal Large Language Models (Chinchilla, Hoffmann et al., 2022): DeepMind’s 2022 paper finding that parameters and training tokens should grow in equal proportion with compute, and that Gopher-era models were undertrained.
- Source: Training Language Models to Follow Instructions with Human Feedback (InstructGPT): OpenAI’s 2022 paper establishing the SFT → reward model → PPO recipe for RLHF.
- Source: Transformers docs: Text generation and Chat templates: Transformers guides on generate()/GenerationConfig defaults and pitfalls (20-token limit, greedy default, left padding) and on chat templates (Jinja, add_generation_prompt, duplicate special tokens).
- Source: Transformers v5: Simple model definitions powering the AI ecosystem (Hugging Face blog): Hugging Face’s announcement of Transformers v5: PyTorch-only, modular model definitions, unified tokenizers, built-in serving, first-class quantization and interop with vLLM/SGLang/llama.cpp/MLX.
- Source: Tree of Thoughts: Deliberate Problem Solving with Large Language Models: Yao et al. (2023): generalise CoT into a search over a tree of intermediate thoughts with self-evaluation, lookahead and backtracking.
- Source: Tulu 3 (Lambert et al., 2024) — RLVR: Ai2’s open post-training recipe that named Reinforcement Learning with Verifiable Rewards: RL against a correctness check instead of a learned reward model.
- Source: Turning Up the Heat: Min-p Sampling (Nguyen et al., 2025): Proposes min-p, a truncation sampler whose cut-off scales with the top token’s probability, improving quality at high temperature.
- Source: UMBRELA: An Open-Source Reproduction of the Bing Relevance Assessor (Upadhyay et al.): Open-source GPT-4o relevance labeller reproducing Thomas et al.; low per-label agreement with TREC assessors but near-identical system rankings.
- Source: Understand and count tokens (Gemini API docs): Google’s documentation on Gemini tokens: characters per token, counting endpoints, multimodal token rates and context limits.
- Source: Understanding R1-Zero-Like Training (Dr. GRPO): 2025 Sea AI Lab / NUS paper showing GRPO’s normalisation biases inflate response length, proposing Dr. GRPO, and questioning how much ’emergent’ reasoning comes from the base model.
- Source: Universal and Transferable Adversarial Attacks on Aligned Language Models (GCG): Zou et al. (2023): greedy plus gradient search finds adversarial suffixes on open models that transfer to ChatGPT, Bard and Claude.
- Source: Unsloth Faster MoE training docs: Unsloth’s MoE fine-tuning guide: grouped-GEMM and split-LoRA kernels, VRAM figures for gpt-oss and Qwen3 MoE, router frozen by default, and no 4-bit QLoRA for MoE.
- Source: Unsloth Reinforcement Learning (GRPO) Guide: Unsloth’s guide to running GRPO/RLVR on a single GPU: reward-function design, VRAM numbers, step counts and vLLM-backed generation.
- Source: Unsloth — LoRA Hyperparameters Guide: Unsloth’s practical guide to choosing LoRA/QLoRA hyperparameters, plus its speed and memory claims.
- Source: Unstructured Docs on Chunking: Unstructured’s element-aware chunking: basic and by_title strategies, hard/soft size limits, overlap, table isolation, and recovering original elements from a chunk.
- Source: Unstructured Docs on Partitioning, Elements and Hi-Res Models: Unstructured’s open-source documentation on partition strategies (auto, fast, hi_res, ocr_only), the element types and metadata it emits, and the layout models behind hi_res.
- Source: Using Agent Skills with the API (Claude Platform docs): How to attach Anthropic and custom skills to Messages API calls: container parameter, code execution tool, versions, limits, container reuse and pause_turn.
- Source: Using LLM-as-a-Judge for evaluation (Hamel Husain): Practitioner guide to ‘critique shadowing’: building a binary LLM judge aligned to one domain expert’s pass/fail critiques.
- Source: vCache, verified semantic prompt caching (Schroeder et al., 2025): Shows that a single static similarity threshold cannot bound a semantic cache’s error rate, and proposes per-entry thresholds learned online with a user-set error bound; ICLR 2026.
- Source: Vector Databases Compared (Notebook tutorial): The owner’s guide to vector databases: core concepts, the chunk-embed-upsert-query flow, a comparison of Chroma, FAISS, Qdrant, Weaviate, Milvus, Pinecone, pgvector and Redis, with code.
- Source: Vercel AI Gateway docs (overview, model fallbacks, budgets): Vercel’s hosted multi-provider gateway: OpenAI- and Anthropic-compatible endpoints, zero token markup, provider order plus model fallbacks with per-attempt metadata, and stacked soft-cap budgets.
- Source: Vercel Connect guide and EMA changelog: Vercel’s knowledge-base guide to Connect, a managed broker that mints short-lived, scoped third-party tokens for apps or specific users via Vercel OIDC, plus its October 2026 Enterprise-Managed Authorization beta.
- Source: Vercel Sandbox docs (overview, security model, pricing and quotas): Vercel Sandbox: Firecracker microVMs with root, persistent sessions up to 24h, Active-CPU billing, an egress firewall with TLS-terminating credential injection, and plan quotas.
- Source: verl Documentation: Docs for verl, the open-source HybridFlow-based RL post-training framework used for large-scale GRPO/DAPO/PPO runs.
- Source: Vertex AI Model Garden — Explore Models: Google’s catalogue for discovering, tuning and deploying first-party, partner and open models, with self-deploy serving options.
- Source: Vertex AI RAG Engine — Overview and Transformations: Google’s managed RAG pipeline (corpus, ingestion, chunking, embedding, retrieval), with its region, CMEK and VPC-SC constraints.
- Source: Vertex AI — Consumption Options: The five ways to pay for Gemini on Vertex AI: Standard, Priority and Flex PayGo, Provisioned Throughput and Batch.
- Source: Vertex AI — Generative AI and Data Governance: What Google stores and for how long when you call Gemini and partner models: training restriction, caching, abuse logging, grounding and zero data retention.
- Source: Vertex AI — Generative AI Locations and Endpoints: Global, multi-region (us/eu) and regional endpoints for Gemini on Vertex AI, and what each implies for data residency and features.
- Source: Vertex AI — Provisioned Throughput (Overview and GSU Sizing): Vertex AI’s fixed-term reserved capacity, measured in generative AI scale units (GSUs) with per-model burndown rates.
- Source: Vertex AI — Standard PayGo and Usage Tiers: How Gemini’s shared-capacity pay-as-you-go works: spend-based org-level TPM tiers, 429 semantics and endpoint choice.
- Source: Vertex AI — Tune Gemini Models: Google Cloud’s overview of supervised, preference and reinforcement tuning for Gemini models.
- Source: Vertex AI — VPC Service Controls: How a VPC-SC perimeter blocks exfiltration from Vertex AI / Agent Platform, which features break inside it, and how PSC and Private Google Access fit.
- Source: Vespa blog 2024: Scaling ColPali to billions of PDFs: Vespa post showing binary quantization of ColPali patch vectors (32x smaller, about 16 KB per page) with Hamming-distance MaxSim and optional float re-ranking.
- Source: ViDoRe V3 (Loison et al., ACL 2026): End-to-end multimodal RAG benchmark over 26k enterprise document pages: visual and late-interaction retrievers beat text ones, but a text pipeline with a strong text reranker scores best overall.
- Source: vLLM docs: Automatic Prefix Caching design: Design doc for vLLM’s hash-based KV block reuse across requests.
- Source: vLLM docs: Batch Invariance: vLLM’s documentation for its beta batch-invariant mode (VLLM_BATCH_INVARIANT=1): supported hardware, tested models, tensor-parallel caveats and cost.
- Source: vLLM docs: Data Parallel Deployment: Official guide to vLLM data parallelism: per-rank engine cores, MoE step synchronization, and internal, hybrid or external load balancing.
- Source: vLLM docs: Disaggregated Prefilling: Official, experimental vLLM feature that runs prefill and decode in separate instances linked by a KV-transfer connector.
- Source: vLLM docs: Engine Arguments: Official reference of vLLM engine/server flags and their defaults.
- Source: vLLM docs: Online Serving: Overview of vLLM’s HTTP server: the endpoint families it exposes and their caveats.
- Source: vLLM docs: OpenAI-Compatible Server: Official reference for vLLM’s OpenAI-compatible API: auth, extra sampling parameters, structured outputs, generation-config defaults.
- Source: vLLM docs: Optimization and Tuning: Official vLLM guide to preemption, chunked prefill, scheduler budgets and parallelism settings.
- Source: vLLM docs: Parallelism and Scaling: Official guide to choosing tensor and pipeline parallel sizes and running vLLM across nodes.
- Source: vLLM docs: Quantization: Official list of vLLM’s quantization methods and hardware support matrix.
- Source: vLLM docs: Quantized KV Cache: Official guide to storing vLLM’s KV cache in FP8 and calibrating its scales.
- Source: vLLM docs: Reproducibility: vLLM’s statement of what it does and does not guarantee about reproducible outputs: default seed 0, offline multiprocessing switch, batch invariance for online serving.
- Source: vLLM docs: Speculative Decoding: Official vLLM guide to its speculative decoding methods, configuration and when they help.
- Source: vLLM docs: V1 User Guide: Official guide to the vLLM V1 engine’s goals, removed V0 features and behavior differences.
- Source: vLLM Expert Parallel Deployment docs: vLLM’s guide to serving MoE models with expert parallelism: EP sizing, DP attention, all-to-all backends, the EPLB load balancer and disaggregated prefill/decode.
- Source: vLLM GitHub README: Project README for vLLM: origin at UC Berkeley’s Sky Computing Lab, feature list, hardware coverage, license.
- Source: vLLM release notes v0.8.0 and v0.11.0: GitHub release notes marking V1 as the default engine (v0.8.0, March 2025) and the complete removal of V0 (v0.11.0, October 2025).
- Source: vLLM Structured Outputs Docs: vLLM documentation for grammar-constrained generation: the structured_outputs parameters (choice, regex, json, grammar, structural_tag), the xgrammar/guidance backends, and reasoning-parser interaction.
- Source: vLLM V1: A Major Upgrade to vLLM’s Core Architecture: vLLM team blog post (January 2025) announcing the V1 engine rewrite: separate engine-core process, unified token-budget scheduler, zero-overhead prefix caching, piecewise CUDA graphs.
- Source: Voice AI & Voice Agents: An Illustrated Primer: Practitioner primer from the Pipecat ecosystem on building voice agents: latency budget table, turn detection, interruptions, WebRTC vs WebSockets, speech-to-speech vs cascaded pipelines.
- Source: Voyage AI text embeddings documentation: Voyage AI’s reference for the voyage-4 embedding family: context length, Matryoshka dimensions, quantized output types, input_type prompts and batch limits.
- Source: Wang et al. 2023: Query2doc: EMNLP 2023 paper (Microsoft) that expands a query with an LLM-written pseudo-document, improving BM25 by 3-15% without any fine-tuning.
- Source: We Have a Package for You! (package hallucinations): Spracklen et al. (2024): code LLMs recommend non-existent packages (5.2% commercial, 21.7% open-source), a new package-confusion supply-chain attack.
- Source: Weaviate docs: Hybrid search: Weaviate documentation for hybrid BM25 + vector queries: alpha weighting, fusion methods and thresholds.
- Source: Weaviate vector indexing, filtering and quantization docs: Weaviate’s concept docs on its HNSW, flat, dynamic and HFresh indexes, its ACORN filtering strategy, and PQ, BQ, SQ and RQ compression.
- Source: Web search tool (OpenAI guide): OpenAI’s hosted web search tool for the Responses API: tool variants, domain filters, location, context size, citations, sources and limits.
- Source: Wei et al. 2025: BrowseComp: OpenAI benchmark of 1,266 hard-to-find, easy-to-verify questions for browsing agents, showing deep research far ahead of browsing chat models and gains from more test-time compute.
- Source: Welcome to ADK 2.0 (ADK docs): ADK 2.0 release notes: GA dates, the graph-based Workflow Runtime, breaking changes from 1.x and deprecations.
- Source: What We Learned from a Year of Building with LLMs: Practitioner essay (Yan, Bischof, Frye, Husain, Liu, Shankar, 2024) on tactics, operations and strategy for taking LLM demos to dependable products.
- Source: Who Validates the Validators? (Shankar et al., 2024): HCI paper on EvalGen and ‘criteria drift’: grading outputs changes what people think the criteria should be.
- Source: Why Do Multi-Agent LLM Systems Fail? (Cemri et al., 2025): Berkeley paper deriving MAST, a 14-mode failure taxonomy for multi-agent systems, by grounded-theory coding of execution traces.
- Source: Why Language Models Hallucinate (Kalai et al., 2025): OpenAI/Georgia Tech paper arguing hallucinations are a statistical consequence of pretraining and persist because benchmarks reward guessing.
- Source: Why we built ADK 2.0 (Google Developers Blog): Google’s rationale for ADK 2.0: move orchestration out of the LLM into deterministic graph and dynamic workflows.
- Source: Why we no longer evaluate SWE-bench Verified (OpenAI, 2026): OpenAI’s February 2026 post dropping SWE-bench Verified because of training-data contamination and flawed tests, recommending SWE-bench Pro instead.
- Source: Will We Run Out of Data? (Villalobos et al., Epoch AI, 2024): Epoch AI projection that frontier training sets will match the whole stock of public human text between 2026 and 2032, sooner if models are over-trained.
- Source: WizardLM / Evol-Instruct (Xu et al., 2023): Microsoft paper that uses an LLM to rewrite seed instructions into progressively harder ones (Evol-Instruct) to build complex instruction data.
- Source: Work with sessions (Claude Agent SDK docs): How the Claude Agent SDK persists conversations as local JSONL transcripts and how continue, resume and fork work.
- Source: Workflow concepts and orchestrations (Microsoft Agent Framework docs): Agent Framework workflow APIs (graph WorkflowBuilder and functional), runtime primitives, and built-in multi-agent orchestrations.
- Source: Workflow SDK docs: workflows, steps, hooks and webhooks (Vercel): Docs for Vercel’s Workflow SDK: "use workflow" / "use step" directives, event-log replay, default step retries, and hooks/webhooks for human-in-the-loop suspension.
- Source: Writing effective tools for agents (Anthropic, 2025): Anthropic engineering post on designing and evaluating agent tools: consolidation, token-efficient responses, truncation, namespacing and eval-driven iteration.
- Source: XGrammar: Dong et al. (2024, MLSys ‘25): a grammar engine that makes CFG-constrained decoding near-zero overhead by prechecking context-independent tokens.
- Source: XGrammar (GitHub README and docs): MLC’s XGrammar project page: Apache-2.0 structured-generation engine (JSON, regex, EBNF/Lark CFGs, structural tags) that is the default backend in vLLM, SGLang, TensorRT-LLM and MLC-LLM; XGrammar-2 released May 2026.
- Source: XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in LLMs: NAACL 2024 test suite of 250 safe-but-scary-sounding prompts that measures over-refusal.
- Source: Yan et al. 2024: Corrective Retrieval Augmented Generation (CRAG): Paper adding a lightweight retrieval evaluator that triggers refine, web-search or both depending on confidence.
- Source: YaRN: Efficient Context Window Extension (Peng et al., 2023): A RoPE-scaling method that extends a model’s usable context with a small amount of fine-tuning.
- Source: Zheng et al. 2023: Step-Back Prompting: ICLR 2024 paper (Google DeepMind) that has the model first ask a more abstract ‘step-back’ question, then reason or retrieve from the principles it surfaces.
- Source: τ-bench: Tool-Agent-User Interaction Benchmark (Yao et al., 2024): Benchmark of tool-using customer-service agents talking to an LLM-simulated user, graded on final database state, introducing pass^k.
- Source: τ²-bench: Evaluating Conversational Agents in a Dual-Control Environment (Barres et al., 2025): τ-bench successor where both user and agent act on a shared world with tools, starting with a telecom domain.