GraphRAG

RAG over an LLM-built knowledge graph with hierarchical community summaries, built for multi-hop and whole-corpus questions that flat top-k retrieval cannot answer; plus cheaper variants like LazyGraphRAG.

Contents

GraphRAG is a family of RAG methods that index a corpus as a graph of entities and relationships instead of (or as well as) a flat list of chunks. In Microsoft’s formulation, an LLM extracts the graph, the graph is clustered into a hierarchy of communities, and each community is summarized ahead of time. Queries can then walk relationships (multi-hop) or map-reduce over community summaries to answer global questions such as “what are the main themes in this dataset?”, which plain top-k retrieval cannot assemble because no single chunk holds the answer (Edge et al., Notebook).

Indexing pipeline

flowchart TD
  A["Documents"] --> B["TextUnits: 600-token chunks, 100 overlap"]
  B --> C["LLM extracts entities, relationships, claims (+ gleaning passes)"]
  C --> D["Knowledge graph"]
  D --> E["Hierarchical Leiden clustering: C0 (root) ... C3 (leaf)"]
  E --> F["LLM writes a summary report per community, bottom-up"]
  F --> G["Index: graph + community summaries + chunk embeddings"]

Sources: Edge et al., Microsoft docs.

  • Extraction. An LLM reads each chunk and outputs entities, relationships and key claims. Chunk size matters: at 600 tokens GPT-4 found almost twice as many entity references as at 2,400 (Edge et al.).
  • Gleanings. After extraction, the LLM is asked whether it missed entities and, if so, prompted to find them, up to a set number of rounds. This recovers recall with larger chunks (Edge et al.).
  • Communities. Leiden clustering is applied recursively until leaf communities can’t be split further, giving levels C0 (root, fewest summaries) through C3 (leaf, most) (Edge et al.).
  • Community summaries are generated bottom-up so higher levels summarize lower ones (Microsoft docs).

Query modes

ModeBest forHow it works
Global search“Summarize the corpus”, themes, trendsMap: chunks of shuffled community summaries each produce a partial answer with a 0-100 helpfulness score; zero scores dropped. Reduce: highest-scoring partials packed into an 8k-token context for the final answer (Edge et al.)
Local search“Everything about entity X and its connections”Start at entities named in the query, fan out to neighbours, relationships, claims and linked chunks (Microsoft docs)
DRIFT searchEntity questions that need broader contextLocal search enriched with community information (Microsoft docs)
Basic searchSimple lookupsPlain top-k vector RAG baseline (Microsoft docs)

Evidence

On two corpora (podcast transcripts about 1M tokens, news about 1.7M tokens), global GraphRAG beat vector RAG on comprehensiveness (72-83% win rates) and diversity (62-82%), judged by an LLM (LLM-as-judge) and cross-checked with claim counting. Vector RAG still won on directness. Root-level (C0) summaries needed over 97% fewer tokens than map-reducing the raw text while still winning 72% on comprehensiveness (Edge et al.). Limits of the evidence: only two corpora, LLM judges, and fabrication rates not measured (Edge et al.).

Cost and LazyGraphRAG

The big cost is indexing: an LLM pass over the entire corpus, far more expensive than embedding chunks, and the graph must be kept fresh (Notebook). Microsoft’s LazyGraphRAG (Nov 2024) attacks this (LazyGraphRAG):

GraphRAGLazyGraphRAGVector RAG
Index-time extractionLLM entity/relation extraction + LLM summariesNLP noun-phrase extraction, no LLM summariesEmbeddings only
Index costHighestAbout vector-RAG cost, about 0.1% of GraphRAGLow
Query-time LLM workMap-reduce over precomputed summariesBest-first + breadth-first community search with an LLM relevance assessor, under a “relevance test budget”Single generation
Reported qualityStrong on globalMatches GraphRAG global at over 700x lower query cost; at 4% of its query cost, beat all compared methods on local and global queriesStrong on local lookups

LazyGraphRAG results come from Microsoft’s own evaluation on 5,590 AP news articles with 100 synthetic queries and LLM judging (LazyGraphRAG). The Notebook also mentions LightRAG as a lighter variant (Notebook).

Operational notes

  • Prompt tuning is expected. Microsoft strongly recommends tuning the extraction prompts to your domain; default prompts may not give the best results (Microsoft docs).
  • Upgrades are disruptive. Minor version bumps require graphrag init --force, which overwrites config and prompts, so back them up (Microsoft docs).
  • Extraction quality drives everything (Notebook). Synthesis: entity resolution (merging “IBM” and “International Business Machines”) is a frequent silent failure that fragments the graph.

Failure modes

  • Indexing bill and time surprise on large corpora (Synthesis from the cost description above).
  • Overkill for fact lookups, where vector RAG is cheaper and more direct (Notebook, Edge et al.).
  • Staleness: incremental updates to the graph and community summaries are harder than upserting a vector (Notebook).
  • Graph-based agentic RAG systems face scalability limits and dependence on high-quality graph data (Singh et al.).

When to use / when not

  • Use for connected, entity-rich corpora (investigations, research collections, org knowledge) where questions span many documents or need a “big picture” (Notebook).
  • Try LazyGraphRAG-style approaches first when budget is tight or the corpus changes (LazyGraphRAG).
  • Don’t use it as the default for lookup-style Q&A; start with hybrid search + reranking and add a graph only when evals show multi-hop or global questions failing (Notebook).
  • Synthesis: if the data is already structured (a CRM, an ERP), query it directly with text-to-SQL rather than re-deriving a graph from prose.

Related: agentic RAG (graph-based agentic architectures), chunking.

Sources

Suggest changes →