Antenor Blog 5 min read
RAG vs GraphRAG: When Knowledge Graphs Actually Pay Off
What GraphRAG adds over vector RAG, what graph construction really costs, and the query patterns where a knowledge graph earns its keep in production.
Vector RAG answers questions whose evidence sits in one or two passages. GraphRAG builds a knowledge graph over your corpus first, which lets the system answer multi-hop and whole-corpus questions that vector search structurally cannot. The trade is a significant upfront extraction cost and an ongoing maintenance burden. The honest rule: start with well-tuned vector RAG, and move to GraphRAG only when your query log proves you need relationship-aware retrieval.
Most teams asking about GraphRAG do not have a GraphRAG problem. They have a retrieval quality problem that better chunking, hybrid search, and reranking would fix at a tenth of the cost. But a real minority do have questions that no amount of vector tuning will answer, and for them the graph is not a luxury. This post is the decision framework we use with clients before writing any code.
What vector RAG actually does
Standard RAG splits documents into chunks, embeds them, and retrieves the top-k most similar chunks for a query. The LLM then answers from those chunks. This works remarkably well when three conditions hold:
- The answer is localized: the evidence lives in a handful of contiguous passages.
- The question is semantically close to the text that contains the answer.
- The corpus is flat: documents matter individually, not through their relationships.
A support knowledge base, a policy manual, a product documentation set. For these, vector RAG plus a reranker is usually the right stopping point.
Where vector RAG structurally fails
Two query families break the localized-evidence assumption, and no embedding model fixes them.
Multi-hop questions
“Which suppliers of our tier-1 vendors were affected by the port closure?” requires joining facts that live in different documents: a supplier list here, a vendor relationship there, a news item somewhere else. Top-k similarity retrieves passages similar to the question, not the chain of entities that connects the answer. The retriever brings back fragments about ports and fragments about vendors, and the model hallucinates the join.
Whole-corpus questions
“What are the recurring failure themes across five years of maintenance reports?” has no single relevant passage. Every document is weakly relevant. Vector search returns an arbitrary sample, and the answer reflects that sample rather than the corpus.
What GraphRAG changes
GraphRAG runs an extraction pass over the corpus, typically LLM-driven, to build a graph of entities and relationships, often with community detection and pre-computed summaries layered on top. At query time the system can traverse relationships for multi-hop questions and use community summaries for corpus-level questions.
That capability is real. So are the costs:
- Construction cost. Entity and relation extraction means one or more LLM calls over every chunk of the corpus. For large corpora this is the dominant line item, and re-runs (schema changes, prompt fixes) multiply it.
- Extraction quality ceiling. The graph is only as good as the extractor. Ambiguous entity resolution (is “Acme Corp” the same node as “Acme Corporation Ltd”?) silently corrupts multi-hop answers.
- Freshness. Documents change. Incremental graph updates are harder than re-embedding a chunk, and a stale graph is worse than no graph because it answers confidently from outdated structure.
- Operational surface. You now run a graph store, an extraction pipeline, and a retrieval layer that mixes traversal with vector search. That is a system, not a library call.
The decision table
| Signal in your workload | Vector RAG | GraphRAG |
|---|---|---|
| Answers live in 1-3 passages | Yes | Overkill |
| Multi-hop questions across documents | Fails | Yes |
| ”Summarize themes across everything” queries | Fails | Yes |
| Entity-heavy corpus (people, parts, contracts, suppliers) | Partial | Yes |
| Corpus changes daily | Yes | Costly to keep fresh |
| Corpus is mostly static reference material | Yes | Yes, cost amortizes well |
| Budget for a one-time heavy indexing pass | Not needed | Required |
| Retrieval quality issues on single-hop questions | Fix RAG first | Wrong tool |
The last row matters most. GraphRAG on top of a badly tuned retrieval stack inherits every one of its problems and adds new ones.
A staged path that avoids the expensive mistake
- Instrument first. Log queries and failures on your current RAG system for a few weeks. Classify failures: retrieval miss, chunking artifact, multi-hop, corpus-level.
- Exhaust the cheap fixes. Hybrid search (BM25 plus vectors), better chunk boundaries, metadata filters, reranking. In our experience this resolves the majority of complaints attributed to “RAG doesn’t work.”
- Quantify the residual. If multi-hop and corpus-level questions remain a material share of real usage, you have a GraphRAG case. If they are rare, handle them with a targeted structured index over the few entity types that matter, not a full graph.
- Scope the graph narrowly. Extract only the entity and relation types your query log demands. A small, accurate graph beats an ambitious, noisy one.
- Budget for maintenance before you build. Decide who owns extraction quality and how updates flow. If nobody owns it, do not build it.
We cover this stack in depth on our RAG and knowledge systems service page.
Frequently asked questions
Is GraphRAG always more accurate than vector RAG?
No. On single-hop questions a well-tuned vector pipeline is typically as accurate and much cheaper. GraphRAG wins specifically on multi-hop and corpus-level questions, and only when extraction quality is high. A noisy graph can be less accurate than plain vector retrieval.
How much does GraphRAG cost to build compared to vector RAG?
Expect roughly an order of magnitude more in indexing compute, because every chunk goes through LLM extraction rather than a single embedding call, plus the engineering time for the graph store and pipeline. Exact numbers depend on corpus size, schema complexity, and the model used for extraction, which is why we scope it per project rather than quoting a flat figure.
Can I combine both?
Yes, and production systems usually do. Vector retrieval handles localized questions, graph traversal handles relational ones, and a router or the retrieval layer itself decides which path a query takes. The hybrid is the norm, not the exception.