GraphRAG
Why it matters
GraphRAG targets the two best-known weaknesses of plain RAG:
- Global questions that require a view of the whole collection: themes, trends, summaries.
- Multi-hop questions that require linking facts from different documents: A relates to B, and B relates to C.
It's most useful for investigative and analytical work over large private collections, such as legal discovery, intelligence analysis, scientific literature and enterprise knowledge bases.
Deep Dive
Standard RAG cuts documents into chunks, embeds them and retrieves the few chunks most similar to the question. That works well when the answer sits in one or two passages. It fails on questions like "What are the main themes across these 5,000 reports?" or "How is this supplier connected to that lawsuit?", because no single chunk contains the answer.
GraphRAG adds structure before retrieval. An LLM reads the documents and extracts the entities (people, companies, products, concepts) and the relationships between them, building a knowledge graph. Retrieval can then follow connections instead of relying on text similarity alone. The term is used generically, and also names a specific open-source system released by Microsoft Research in 2024.
How Microsoft's GraphRAG Works
Indexing:
- Split the documents into chunks.
- Have an LLM extract entities, relationships and key claims from every chunk.
- Merge them into one graph, combining duplicate entities.
- Run a community detection algorithm to group closely connected entities into clusters, at several levels of detail.
- Have an LLM write a summary report for each community.
Querying:
- Global search: for broad questions, the system gives the question to many community summaries, collects partial answers and combines them, map-reduce style.
- Local search: for questions about specific entities, it starts from the matching entities and pulls in their neighbors, relationships, community reports and source text.
The Cost
Indexing is expensive. Every chunk passes through an LLM, often several times, and the summaries add more calls. Indexing a large collection can cost hundreds or thousands of times more than computing embeddings. Updates are awkward too, since new documents can change the graph and its summaries. Extraction quality also matters: missed or wrongly merged entities quietly degrade answers.
Lighter variants address this. Microsoft's LazyGraphRAG postpones most LLM work until query time, and projects such as LightRAG and HippoRAG use cheaper graph construction. Graph databases and frameworks such as Neo4j and LlamaIndex offer their own graph-based retrieval tools.
When to Use It
Use GraphRAG when users really ask global or connect-the-dots questions and the collection is fairly stable. Don't use it for straightforward fact lookup, where hybrid search with a reranker is cheaper, faster and usually just as good. A sensible path is to build plain RAG first, collect the questions it fails on, and check whether they're the kind a graph would fix.
An Alternative Route
Agentic retrieval solves some of the same problems differently: instead of pre-building a graph, an agent searches repeatedly, following leads from one document to the next. It costs more per query but nothing up front, and many teams now try it before investing in a graph.