Rooftop

RAG 与 Agentic RAG

12分钟阅读

1. What Is RAG?

RAG stands for Retrieval-Augmented Generation.

Its purpose is to address a fundamental limitation of large language models: a model may not know the latest information, private data, or domain-specific knowledge. Before generating an answer, a RAG system retrieves relevant information from an external knowledge source and provides it to the model as context.

A standard RAG pipeline can be summarized as:

User query
→ Retrieve relevant knowledge
→ Add the retrieved knowledge to the model context
→ Generate an answer

For example, suppose the user asks:

What does Depends do in FastAPI?

Instead of relying only on the model’s internal knowledge, the system first retrieves relevant content from the FastAPI documentation or an internal knowledge base.

The model may receive a prompt like this:

Reference material:
FastAPI uses Depends to declare dependencies.
Dependencies can be used for database sessions, authentication,
and shared request parameters.
 
Question:
What does Depends do in FastAPI?

The model then generates an answer based on the retrieved material.

The key idea is that RAG does not permanently teach the model new knowledge. Instead, it:

Provides external evidence to the model at inference time.

2. Core Components of RAG

A complete RAG system usually contains two stages:

Knowledge base construction
Online retrieval and generation

2.1 Knowledge Base Construction

The original data may come from:

  • Markdown, PDF, or Word files
  • Official documentation
  • Databases
  • GitHub repositories
  • User history
  • Internal company documents

These documents are usually too large to retrieve as a whole. They are therefore divided into smaller text units called chunks.

For example, a FastAPI document may be divided into:

Chunk 1: Dependency injection in FastAPI
Chunk 2: Basic usage of Depends
Chunk 3: Database session lifecycle

Chunking improves retrieval precision.

If a chunk is too large, it may contain too much irrelevant information. If it is too small, it may lose important context. Chunk design therefore has a direct effect on RAG quality.

Each chunk is then converted into a numerical vector by an embedding model and stored in a vector database.

Text chunk
→ Embedding model
→ Vector
→ Vector database

Common storage options include:

  • PostgreSQL with pgvector
  • Qdrant
  • Milvus
  • Pinecone
  • Elasticsearch
  • FAISS

An embedding vector represents the semantic meaning of a piece of text. Texts with similar meanings usually have vectors that are close to one another.

2.2 Online Retrieval

When a user submits a question, the system also converts the question into a vector and searches the knowledge base for the most similar chunks.

For example:

Question:
How can FastAPI inject a database connection?

The retriever may return:

FastAPI can use Depends to inject a database session.

The system then places the retrieved content into the prompt and sends it to the language model.

This process consists of three main steps:

Retrieve
Augment
Generate
  • Retrieve: Find relevant external information.
  • Augment: Add that information to the model context.
  • Generate: Produce an answer based on the augmented context.

RAG is often discussed together with embeddings and vector databases, but they are not equivalent.

RAG means:

Retrieving external information and using it to support generation.

The retrieval mechanism does not have to be vector search. It may also use:

  • SQL queries
  • Keyword search
  • BM25 full-text search
  • Graph database queries
  • API calls
  • Web search
  • Code search

For example, in Sisyphus, the system may need to identify the user’s weakest knowledge areas. This can be done directly with SQL:

SELECT weakness_tag, AVG(score)
FROM answer_records
GROUP BY weakness_tag
ORDER BY AVG(score) ASC;

The query result can then be passed to the model to generate the next question.

This is still RAG because the model is using externally retrieved information. The retrieval source is simply a relational database rather than a vector database.

In practice, different data types should use different retrieval methods:

Unstructured documents → Vector search or full-text search
Structured business data → SQL
Exact class names or error codes → Keyword search
Relationship-heavy data → Graph database

4. Limitations of Standard RAG

A standard RAG pipeline is usually fixed:

User question
→ Retrieve once
→ Return Top K documents
→ Generate an answer

This approach is simple and stable, but it has several limitations.

4.1 The User Query May Be Poor for Retrieval

A user may ask:

Why is this not working?

Without conversational context, this query is almost impossible to retrieve against effectively.

The system may need to rewrite it as:

Why is an asynchronous database session not closed after a FastAPI request finishes?

This process is called query rewriting.

4.2 The First Retrieval Attempt May Fail

The retrieved content may be:

  • Irrelevant
  • Incomplete
  • Missing a key section
  • Contradictory

A standard RAG pipeline often does not evaluate whether the retrieved evidence is sufficient. It simply proceeds to generation.

This creates a common failure mode:

Retrieval is wrong, but the model still produces a confident answer.

4.3 A Fixed Retrieval Strategy Lacks Adaptability

Different questions require different sources.

For example:

“What is the official definition of FastAPI Depends?”
→ Search technical documentation
 
“What topics have I repeatedly misunderstood?”
→ Query user learning history
 
“Why does this API return HTTP 500?”
→ Search logs and source code

A standard RAG system often uses a fixed retriever or a single knowledge base. It may not dynamically select the most appropriate source.

5. What Is Agentic RAG?

Agentic RAG can be defined as:

A RAG system in which an agent dynamically controls the retrieval process.

In standard RAG, the retrieval logic is predefined by application code. In Agentic RAG, the model can participate in decisions such as:

  • Whether retrieval is needed
  • Which data source should be used
  • What query should be issued
  • Whether the query should be rewritten
  • Whether another retrieval attempt is necessary
  • Whether the current evidence is sufficient
  • When the process should stop

A typical Agentic RAG workflow looks like this:

User question
→ Decide whether retrieval is needed
→ Select a data source
→ Generate a retrieval query
→ Execute retrieval
→ Evaluate whether the result is sufficient
→ Rewrite the query or switch sources if necessary
→ Generate an answer based on the evidence

The central difference is not simply that Agentic RAG retrieves more often.

The real difference is:

Retrieval changes from a fixed operation into a decision-driven, iterative, and self-correcting process.

6. Core Components of Agentic RAG

6.1 Router

A router determines which path a question should follow.

For example:

Question: What are my weakest recent topics?
→ Query the learning history database
 
Question: What is the official definition of FastAPI Depends?
→ Search the documentation knowledge base
 
Question: What is 1 + 1?
→ No retrieval required

The router may return structured output:

{
  "need_retrieval": true,
  "source": "learning_history"
}

6.2 Query Planner

A complex question may need to be decomposed into multiple subproblems.

For example:

Analyze why I keep struggling with dependency injection and design my next review session.

This request includes several tasks:

Retrieve recent answer history
Identify recurring mistakes
Map the mistakes to knowledge concepts
Retrieve relevant learning material
Generate a review plan

The agent first creates a plan and then executes the required steps.

6.3 Retriever

The retriever is the tool that performs the actual search.

A system may provide several retrieval tools:

search_documents()
query_learning_history()
search_previous_questions()
query_mastery_records()
search_code()

The agent selects the appropriate retriever based on the current task.

6.4 Retrieval Grader

A retrieval grader determines whether the retrieved content is actually relevant.

For example:

Question:
How does FastAPI manage a database session?
 
Retrieved content:
FastAPI automatically generates Swagger UI.

Both pieces of text concern FastAPI, but the retrieved document does not answer the question.

The grader may return:

{
  "relevant": false,
  "reason": "The document does not discuss database sessions."
}

The system can then rewrite the query or switch to another source.

6.5 Grounding Checker

After the model generates an answer, the system may check whether the claims are supported by the retrieved evidence.

For example, suppose the model states:

Depends automatically caches database connections for 24 hours.

If the retrieved material does not support this claim, the grounding checker should flag it as unsupported.

This step is used to reduce hallucinations.

7. Standard RAG vs. Agentic RAG

DimensionStandard RAGAgentic RAG
Retrieval flowFixedDynamic
Whether to retrieveUsually always retrievesDecided by the agent
Data sourcesUsually onePotentially multiple
Number of retrievalsUsually onePotentially several
Query generationOriginal query or fixed rewriteDynamically generated
Result evaluationOften absentCan assess sufficiency
Failure handlingFixed fallbackRetry, rewrite, or switch tools
Latency and costLowerHigher
ControllabilityStrongerMore complex
Best suited forClear, simple tasksComplex, multi-step tasks

Standard RAG behaves like a fixed search pipeline.

Agentic RAG behaves more like a research process: it searches, evaluates the result, adjusts the strategy, and verifies the evidence before answering.

8. Engineering Risks of Agentic RAG

Agentic RAG is more capable, but it introduces significantly more engineering complexity.

8.1 Higher Cost

A single user request may trigger:

One routing decision
Two retrieval calls
One retrieval grading call
One query rewrite
One answer generation call
One grounding check

This means more model calls, tool calls, latency, and token usage.

8.2 Risk of Loops

A poorly designed workflow may repeatedly execute:

Evidence insufficient
→ Retrieve again
→ Still insufficient
→ Retrieve again

Without a clear stopping condition, this can cause request storms.

The system should therefore define strict limits:

MAX_RETRIEVAL_ATTEMPTS = 2
MAX_TOOL_CALLS = 5
MAX_DOCUMENTS = 20

It should also define rules such as:

Stop if two consecutive retrievals produce no new useful evidence.
Return “insufficient evidence” if retrieval confidence remains too low.
Explicitly report unresolved conflicts between sources.

Agentic does not mean unlimited autonomy.

It means:

Dynamic decision-making within explicit operational boundaries.

8.3 More Difficult Debugging

When standard RAG fails, the main components to inspect are usually:

Chunking
Embeddings
Retrieval results
Prompt construction

Agentic RAG adds more possible failure points:

Was the routing decision correct?
Was the wrong tool selected?
Was the query rewritten badly?
Were unnecessary retries performed?
Did the workflow stop too early or too late?

For this reason, Agentic RAG requires strong logging and observability.

9. Applying RAG to Sisyphus

Sisyphus may retrieve four main types of information:

Knowledge material
User answer history
Weakness and mastery data
Current conversation context

Standard RAG Example

When generating a new question, the system may follow a fixed process:

Retrieve knowledge related to the current topic
→ Query the user’s weaknesses
→ Send both to the model
→ Generate a question

For example:

User weakness:
dependency_injection
 
Reference knowledge:
FastAPI uses Depends to declare general-purpose dependencies.
 
Task:
Generate a medium-difficulty question focused on the scope
of dependency injection.

A fixed RAG workflow is sufficient for this case.

Agentic RAG Example

Suppose the user answers:

Depends is used to create database objects.

The system may execute the following process:

Identify that the answer concerns dependency injection
→ Retrieve the expected answer points
→ Retrieve the definition of Depends from learning material
→ Check whether the user has made the same mistake before
→ Determine whether the evidence is sufficient
→ Produce an evaluation and follow-up question

The retrieved evidence may show:

Learning material:
Depends is a general dependency injection mechanism,
not a database-specific tool.
 
History:
The user has made the same database-only interpretation twice before.

The model can then produce a more accurate evaluation:

{
  "accepted": false,
  "score": 55,
  "misconceptions": ["Depends was incorrectly treated as a database-specific feature."],
  "follow_up_question": "What other kinds of dependencies can be injected with Depends?"
}

The value of Agentic RAG here is not simply that it retrieves more information.

Its value is that:

The system decides which evidence is needed based on the current answer.

Sisyphus should not begin with a highly complex Agentic RAG architecture.

A more suitable progression is the following.

Stage 1: Fixed RAG

Start with a deterministic pipeline:

Retrieve reference knowledge
Retrieve user weakness data
Generate a question or evaluation

At this stage, the main concepts to understand are:

  • Chunking
  • Embeddings
  • Retrieval
  • Metadata
  • Prompt context
  • Grounded generation

Stage 2: Retrieval Quality Evaluation

Add a simple decision:

Is the retrieved evidence sufficient?

If not, allow only one query rewrite.

Retrieve
→ Grade
→ Rewrite once
→ Retrieve again
→ Generate

Stage 3: Add a Router

Allow the model to choose from a small set of options:

Knowledge base
User history
Weakness data
No retrieval

The tool set should remain small, and the router output should use structured output.

Stage 4: Build a Bounded Workflow

The final workflow may look like:

route
→ retrieve
→ grade
→ rewrite
→ evaluate
→ verify
→ fallback

Each node should have:

  • Clear input
  • Clear output
  • Limited retries
  • Independent testability
  • Complete logging

At that point, the system becomes a relatively mature Agentic RAG workflow.

Summary

The core idea of RAG is:

Providing external knowledge to a language model so that its answer is grounded in retrieved evidence.

The core idea of Agentic RAG is:

Allowing an agent to decide how to retrieve information, whether more retrieval is needed, and when to stop.

The two approaches are not simple replacements for one another.

Standard RAG is better suited for:

Clear tasks
Fixed data sources
Low latency
High controllability

Agentic RAG is better suited for:

Complex questions
Multiple data sources
Dynamic planning
Retrieval correction

For Sisyphus, the correct path is to first build a reliable fixed RAG pipeline, and then gradually add routing, retrieval grading, bounded retries, and grounding checks.

反向链接