Use graph-based RAG when the application needs explicit relationships, entity-centered exploration or summaries across a connected corpus, and the value justifies building and maintaining that structure. Start with conventional retrieval when relevant passages can answer the question directly. Graph RAG is an additional retrieval architecture, not a guarantee of higher accuracy.
This guide compares the two approaches through an illustrative supplier-risk workload, two architecture flows and a selection matrix. It does not claim measured performance improvements. The right design must be tested against the same permitted sources and representative questions.
Graph RAG vs RAG: define the architecture first
Conventional RAG retrieves relevant source passages, assembles evidence and asks a model to answer from that context. Retrieval can use keywords, vectors, hybrid search, metadata and reranking. Calling it “traditional” does not mean it must be a naive vector-only lookup.
Graph-based RAG incorporates relationships among entities or concepts into retrieval and context construction. The graph may originate in structured business records, human-curated knowledge or model-extracted relations. Those sources have different reliability and maintenance requirements.
Microsoft GraphRAG is a particular implementation that extracts graph information and builds community summaries. Its documented local search combines entity-related structured information with source text. Its global search uses community reports for broader corpus questions. Do not assume every graph-based retrieval product uses that same pipeline.
Architecture 1: passage-centered retrieval
Approved sources + permissions
→ parse and preserve structure
→ chunks + metadata + search index
Authenticated question
→ authorized keyword / vector / hybrid retrieval
→ rerank and assemble permitted evidence
→ generate answer with source references
→ evaluate support, correctness and task usefulness
This architecture is often an effective baseline for policy lookup, support documentation and questions whose evidence is contained in a small set of passages. It can still support multi-step retrieval. The limitation to test is whether the retrieval strategy consistently finds all necessary evidence, not whether the index has a graph label.
Architecture 2: graph-assisted evidence retrieval
Approved sources + permissions
→ parse and extract candidate entities / relations
→ resolve identities + retain source provenance
→ graph + source passages + optional derived summaries
Authenticated question
→ choose entity, relationship or corpus-level route
→ authorized graph neighborhood / permitted summaries
→ retrieve supporting source passages
→ assemble context and generate a cited answer
→ evaluate graph facts and the final task result
The additional graph path creates opportunities to connect evidence, but also new failure points: incorrect entity merges, invented edges and outdated summaries. Keep underlying sources accessible for verification. A relationship drawn by an extraction model is a candidate assertion until the application has established the required confidence and review process.
An illustrative supplier-risk workload
Imagine an approved corpus containing supplier records, product dependencies, contracts and incident reports. The following examples are fictional application scenarios, not completed client projects or measured benchmarks.
| Question | Useful evidence route | Why | Risk to test |
|---|---|---|---|
| What is the notice period in contract C-104? | Exact identifier plus contract passages | The relevant clause is localized | Wrong contract version or omitted exception |
| Which products depend on a supplier affected by incident I-20? | Verified supplier-to-product relations plus incident evidence | The answer follows relationships across records | Merged supplier identities or missing dependencies |
| What recurring issues appear across supplier incidents? | Permitted corpus summaries and supporting reports | The question asks for aggregation rather than one similar passage | Summary bias or unsupported generalization |
| Which document explains escalation for a late delivery? | Hybrid passage retrieval | Terminology varies but source guidance is local | Overcomplicated graph route without added value |
Some dependency questions are better answered with a structured database query than model-generated graph traversal. If an authoritative system already maintains product-to-supplier links, use that source rather than extracting a less reliable copy from prose. The model can explain the result while deterministic services enforce the query and access scope.
Entity linking is where many graph errors begin
“Acme,” “Acme Ltd” and a supplier ID may refer to one organization, or to different legal entities. Merging them incorrectly can create a plausible but false relationship chain. Conversely, failing to merge aliases can hide a real dependency.
Define stable identifiers and entity types. Preserve the original mention and source reference alongside the resolved identity. Record uncertainty, review rules and conflict handling. Do not overwrite an authoritative identifier with a model’s guess.
Edges need provenance and time
A relationship should carry its source, version and relevant validity period. A supplier that supported a product last year may not support it now. Distinguish a contractual relationship from a speculative mention in an incident narrative.
For consequential answers, require a path back to approved evidence and retain qualifications. A short graph path is not proof of causation. Explain what the underlying records establish and what remains unknown.
Summaries inherit source restrictions
A summary assembled from restricted documents can disclose their content even if the original passages are hidden. Build and retrieve derived material within an appropriate permission scope, or reconstruct permitted evidence for the caller. A mixed-permission community report cannot be made safe merely by hiding its citation links.
See our permission-aware RAG guide for identity propagation, revocation and cache boundaries. Those controls apply to graph nodes, edges and summaries as well as chunks.
Accuracy, cost and maintenance trade-offs
A graph can expose relationships that passage similarity misses. It can also make extraction errors reusable across many answers. Measure both: relationship coverage and relation correctness, followed by final answer support and usefulness.
Additional indexing work may include extraction, identity resolution, graph construction, summary generation and review. Updates can require invalidating several derived artifacts. Estimate these costs separately from query-time retrieval and generation.
Microsoft documents a dedicated GraphRAG indexing pipeline. Treat such a pipeline as a maintained data product: version the extraction settings, source snapshot and derived outputs. Evaluate what must be rebuilt when documents, identities or permissions change.
| Change | Passage-centered concern | Additional graph concern |
|---|---|---|
| Document edited | Reparse and update affected chunks | Refresh extracted relations and dependent summaries |
| Entity renamed | Update searchable text and aliases | Preserve identity and repair resolution mappings |
| Source deleted | Remove chunks and cached answers | Remove unsupported edges and invalidate derived reports |
| Permission revoked | Re-authorize retrieval and caches | Prevent disclosure through graph neighborhoods and summaries |
| Extraction model changed | Potential parsing or embedding changes | Re-evaluate graph facts and extraction consistency |
A practical selection and evaluation matrix
Begin with a strong baseline: appropriate parsing, metadata, hybrid retrieval and source permissions. Do not attribute a weak baseline’s failures to the absence of a graph before diagnosing its retrieval configuration.
| Condition | Starting decision | Evidence required |
|---|---|---|
| Most questions are local lookups | Passage-centered baseline | Good evidence recall and supported answers |
| Questions repeatedly follow cross-record relationships | Evaluate graph or authoritative structured queries | Correct entities, edges and path evidence |
| Users ask broad thematic corpus questions | Evaluate permission-scoped aggregation or graph summaries | Representative coverage and traceable conclusions |
| Source identities are ambiguous | Improve source data and resolution before expanding graph use | Reviewed merges, aliases and conflict handling |
| Permissions change frequently | Prioritize revocation and derived-data controls | Measured synchronization and no forbidden summaries |
Run an experiment that can reject the graph hypothesis
- Label questions by local lookup, relationship traversal and corpus synthesis.
- Freeze the approved sources and caller scopes for both candidates.
- Build a tuned passage baseline and a clearly defined graph-assisted alternative.
- Measure evidence coverage, answer support, entity or relation errors and source freshness.
- Record latency, indexing effort, update cost and review time.
- Test denied access, deletions and conflicting sources.
- Adopt the added graph path only where the observed benefit justifies its ownership burden.
A routed architecture can use passage search for simple questions and graph assistance for relationship-heavy ones. Keep routing observable and allow abstention when the required graph evidence is incomplete. Avoid letting the model silently invent missing edges to complete a path.
Common questions
Is Graph RAG more accurate than traditional RAG?
It can help with relationship-heavy or corpus-wide questions, but accuracy depends on source quality, extraction, identity resolution and evaluation. It can also introduce false relations. Test the actual workload instead of assuming an automatic improvement.
Does Graph RAG replace vector search?
Not necessarily. Graph-assisted systems can use embeddings and passage retrieval alongside graph relationships and summaries. The architecture should choose the evidence route appropriate to the question.
When is ordinary RAG enough?
When well-retrieved permitted passages reliably answer the target questions with acceptable support, freshness and latency, an additional graph may create cost without sufficient benefit.
What is the difference between local and global GraphRAG search?
In Microsoft’s implementation, local search is entity-centered and combines graph information with source text, while global search draws on community reports for broader corpus questions. Other graph-based products may use different terminology.
Can a graph summary leak restricted documents?
Yes, derived summaries can reveal source information. Enforce permissions for those artifacts and their supporting evidence; hiding source links alone does not establish a safe boundary.
Choose the evidence architecture before the tool
Continue with the enterprise RAG architecture guide and RAG versus fine-tuning comparison. For a workload-specific review of retrieval, graphs and source permissions, request an enterprise RAG architecture review.