Share with your CTO
Most enterprise RAG systems, which let AI models answer questions by fetching relevant documents first rather than relying on memory alone, are failing not because the underlying AI is weak but because the document libraries feeding them are insecure, stale, and unmonitored. Researcher and analyst Brendan Tierney’s survey of the production RAG literature identifies four fixable gaps in priority order: permissions aren’t enforced at retrieval time, indexes go stale without freshness SLOs, evaluation stops at launch instead of running continuously, and nobody has priced what each answer actually costs. The full enterprise RAG reliability argument is paired with an open-access survey.
What this means for your business
The organizations most exposed here aren’t the ones still piloting RAG, they’re the ones who shipped it and moved on. A system that passed its launch evaluation, shows green dashboards, and answers fluently is still silently serving stale or improperly permissioned content to users who have no way to know the difference. The gap that decides whether this story is about you is operational maturity: did your team build a retrieval pipeline or a retrieval service, one with ongoing measurement, access controls baked into the index itself, and cost accounting per query?
The security problem is sharper than most teams have priced in. Post-retrieval filtering, the common substitute for real permission enforcement, doesn’t just risk exposing restricted documents. It also means the ranking signals that surface documents have already crossed the permission boundary before the filter runs, and the filter itself becomes a single point of breach. Worse, indirect prompt injection, where malicious instructions are embedded inside a document the system later retrieves and those instructions then hijack the model’s behavior, is ranked the number-one risk in the OWASP Top 10 for LLM applications for two consecutive editions. That threat is structural to how most enterprise RAG is currently built, not a configuration bug.
The GraphRAG cost figure deserves a moment of attention. Microsoft’s own LazyGraphRAG post implies full GraphRAG indexing runs at roughly one thousand times the cost of a standard vector index. That’s not a reason to avoid it, graph-based retrieval genuinely handles multi-hop and thematic questions that chunk-based systems can’t touch. But it is a reason to treat GraphRAG as a targeted tool for specific query types with an explicit budget ceiling, not a default upgrade. If your team is currently evaluating it as a wholesale replacement for your existing retrieval layer, that decision is worth revisiting before the indexing bill arrives.
The practical leading indicator to watch is whether your RAG system has freshness SLOs the way it has latency SLOs. Latency is monitored because someone felt the pain when it degraded. Freshness will get the same treatment only after a costly wrong answer, delivered with full confidence, reaches the wrong person at the wrong moment. The teams that build the measurement discipline before that happens own the difference between a retrieval service and a liability.
Concept deep-dive: Retrieval-Augmented Generation (RAG)
RAG is an AI architecture where, instead of answering purely from patterns learned during training, a language model is first handed a set of freshly fetched documents relevant to the question, then generates its answer using those documents as evidence. Think of it as giving the model an open-book exam rather than a closed-book one. Enterprises adopted it quickly because it made AI answers traceable to specific sources, but that traceability only holds if the sources themselves are current, accurate, and properly access-controlled.
Based on reporting from How to Make Enterprise RAG Systems More Reliable, originally published 2026-11-15 18:19:00.

