GraphRAG
RAG over an LLM-built knowledge graph with hierarchical community summaries, built for multi-hop and whole-corpus questions that flat top-k retrieval cannot answer; plus cheaper variants like LazyGraphRAG.
Contents
GraphRAG is a family of RAG methods that index a corpus as a graph of entities and relationships instead of (or as well as) a flat list of chunks. In Microsoft’s formulation, an LLM extracts the graph, the graph is clustered into a hierarchy of communities, and each community is summarized ahead of time. Queries can then walk relationships (multi-hop) or map-reduce over community summaries to answer global questions such as “what are the main themes in this dataset?”, which plain top-k retrieval cannot assemble because no single chunk holds the answer (Edge et al., Notebook).
Indexing pipeline
flowchart TD A["Documents"] --> B["TextUnits: 600-token chunks, 100 overlap"] B --> C["LLM extracts entities, relationships, claims (+ gleaning passes)"] C --> D["Knowledge graph"] D --> E["Hierarchical Leiden clustering: C0 (root) ... C3 (leaf)"] E --> F["LLM writes a summary report per community, bottom-up"] F --> G["Index: graph + community summaries + chunk embeddings"]
Sources: Edge et al., Microsoft docs.
- Extraction. An LLM reads each chunk and outputs entities, relationships and key claims. Chunk size matters: at 600 tokens GPT-4 found almost twice as many entity references as at 2,400 (Edge et al.).
- Gleanings. After extraction, the LLM is asked whether it missed entities and, if so, prompted to find them, up to a set number of rounds. This recovers recall with larger chunks (Edge et al.).
- Communities. Leiden clustering is applied recursively until leaf communities can’t be split further, giving levels C0 (root, fewest summaries) through C3 (leaf, most) (Edge et al.).
- Community summaries are generated bottom-up so higher levels summarize lower ones (Microsoft docs).
Query modes
| Mode | Best for | How it works |
|---|---|---|
| Global search | “Summarize the corpus”, themes, trends | Map: chunks of shuffled community summaries each produce a partial answer with a 0-100 helpfulness score; zero scores dropped. Reduce: highest-scoring partials packed into an 8k-token context for the final answer (Edge et al.) |
| Local search | “Everything about entity X and its connections” | Start at entities named in the query, fan out to neighbours, relationships, claims and linked chunks (Microsoft docs) |
| DRIFT search | Entity questions that need broader context | Local search enriched with community information (Microsoft docs) |
| Basic search | Simple lookups | Plain top-k vector RAG baseline (Microsoft docs) |
Evidence
On two corpora (podcast transcripts about 1M tokens, news about 1.7M tokens), global GraphRAG beat vector RAG on comprehensiveness (72-83% win rates) and diversity (62-82%), judged by an LLM (LLM-as-judge) and cross-checked with claim counting. Vector RAG still won on directness. Root-level (C0) summaries needed over 97% fewer tokens than map-reducing the raw text while still winning 72% on comprehensiveness (Edge et al.). Limits of the evidence: only two corpora, LLM judges, and fabrication rates not measured (Edge et al.).
Cost and LazyGraphRAG
The big cost is indexing: an LLM pass over the entire corpus, far more expensive than embedding chunks, and the graph must be kept fresh (Notebook). Microsoft’s LazyGraphRAG (Nov 2024) attacks this (LazyGraphRAG):
| GraphRAG | LazyGraphRAG | Vector RAG | |
|---|---|---|---|
| Index-time extraction | LLM entity/relation extraction + LLM summaries | NLP noun-phrase extraction, no LLM summaries | Embeddings only |
| Index cost | Highest | About vector-RAG cost, about 0.1% of GraphRAG | Low |
| Query-time LLM work | Map-reduce over precomputed summaries | Best-first + breadth-first community search with an LLM relevance assessor, under a “relevance test budget” | Single generation |
| Reported quality | Strong on global | Matches GraphRAG global at over 700x lower query cost; at 4% of its query cost, beat all compared methods on local and global queries | Strong on local lookups |
LazyGraphRAG results come from Microsoft’s own evaluation on 5,590 AP news articles with 100 synthetic queries and LLM judging (LazyGraphRAG). The Notebook also mentions LightRAG as a lighter variant (Notebook).
Operational notes
- Prompt tuning is expected. Microsoft strongly recommends tuning the extraction prompts to your domain; default prompts may not give the best results (Microsoft docs).
- Upgrades are disruptive. Minor version bumps require
graphrag init --force, which overwrites config and prompts, so back them up (Microsoft docs). - Extraction quality drives everything (Notebook). Synthesis: entity resolution (merging “IBM” and “International Business Machines”) is a frequent silent failure that fragments the graph.
Failure modes
- Indexing bill and time surprise on large corpora (Synthesis from the cost description above).
- Overkill for fact lookups, where vector RAG is cheaper and more direct (Notebook, Edge et al.).
- Staleness: incremental updates to the graph and community summaries are harder than upserting a vector (Notebook).
- Graph-based agentic RAG systems face scalability limits and dependence on high-quality graph data (Singh et al.).
When to use / when not
- Use for connected, entity-rich corpora (investigations, research collections, org knowledge) where questions span many documents or need a “big picture” (Notebook).
- Try LazyGraphRAG-style approaches first when budget is tight or the corpus changes (LazyGraphRAG).
- Don’t use it as the default for lookup-style Q&A; start with hybrid search + reranking and add a graph only when evals show multi-hop or global questions failing (Notebook).
- Synthesis: if the data is already structured (a CRM, an ERP), query it directly with text-to-SQL rather than re-deriving a graph from prose.
Related: agentic RAG (graph-based agentic architectures), chunking.