Skip to main content
Zubnet AI › Learn › Wiki › Agentic RAG
Using AI

Agentic RAG

Also known as: Agentic Retrieval, Agentic Search
Retrieval-augmented generation where the model itself decides when to search, what to search for and whether it has found enough, instead of following a fixed retrieve-then-answer pipeline.

Why it matters

This is how most modern AI search products work. Deep research features in ChatGPT, Claude, Gemini and Perplexity run dozens or hundreds of searches per question. Coding agents explore a codebase the same way, with repeated searches and file reads instead of one embedding lookup.

It became practical because models got good at tool use and at judging their own results. The payoff is largest on hard questions: multi-hop questions, vague questions that need clarifying, and questions where the first search reveals the right term to search for next.

Deep Dive

Classic RAG is a fixed pipeline: take the user's question, retrieve the top few chunks once, paste them into the prompt and generate an answer. The model has no say in what gets retrieved. If the search misses, the answer is wrong, and nothing in the system notices.

Agentic RAG puts the model in charge of retrieval. Search becomes a tool that the model can call, as many times as it needs. It can rewrite the question into better queries, break a complex question into parts, read the results, notice gaps, search again with what it just learned, and stop when it has enough evidence.

What the Agent Can Do

  • Decide whether to retrieve at all. "What's 2+2" needs no search.
  • Write its own queries. Users write messy questions. The model can turn one into several precise searches.
  • Choose among sources. Vector search, keyword search, SQL, a web search API, file tools and other systems can all be offered as tools.
  • Iterate. Read results, follow references and refine. This is the ReAct loop applied to retrieval.
  • Check itself. Grade whether retrieved passages actually answer the question, and verify the draft answer against the sources.

The Simplest Version

You don't need a framework. Give a capable model one or two search tools with clear descriptions, tell it to search before answering and to cite what it used, and cap the number of steps. That alone often beats a carefully tuned single-shot pipeline on hard questions.

Trade-offs

  • Latency and cost: every search step is another model call. A simple question that took one second can take ten, and a deep research task can take minutes and use hundreds of thousands of tokens.
  • Context growth: results pile up in the context window, so tools should return compact, relevant snippets.
  • Less predictable: the agent may stop too early, search in circles or wander. Evaluation must look at the whole trajectory, not just the answer.
  • Security: retrieved web pages and documents can contain prompt injections, and an agent with other tools can act on them.

Retrieval Quality Still Matters

An agent can only compensate so much for a bad search tool. Good chunking, hybrid search and reranking make each step more useful and reduce the number of steps needed. Interestingly, agents often do well with simple tools, such as keyword search and reading files directly, because they can make up for imprecision by trying again.

When to Use Which

For high-volume, simple lookups where speed matters, such as a support bot answering from a FAQ, classic single-shot RAG is still the right tool. For research, analysis and complex questions, let the model drive. Many systems use both: a quick first retrieval, with the agent searching further only when needed.

← All Terms
ESC