How can we make a RAG (Retrieval-Augmented Generation) more accurate? Developing a RAG is sometimes tricky. For example, I have a huge codebase. I want to know the algorithm in the codebase, and this is my query. I cannot feed the entire codebase to the LLM (Large Language Model). In this case, with conventional RAG, I can extract a few code chunks, but I may not get all the relevant chunks needed to reconstruct the full algorithm. RAG accuracy mainly depends on chunk size and chunk overlap.
In conventional RAG, we have a large dataset corpus. We divide the entire dataset into small chunks. Then we convert each chunk into a vector. When a user queries an AI agent with an LLM, the query is also converted into a vector. Then we compare each chunk vector with the query vector. The best-matched chunks are sorted, and the top K chunks are returned. The embedding model converts the text to a vector.
Many mathematical methods exist to compare vectors or probability distributions. Some very popular ones use cosine similarity, dot product, etc. For example, when two vectors are similar, they point in the same direction, and when they are not similar, they point in opposite directions.
TIP:
To improve RAG accuracy, I often embed short metadata in every chunk, containing important additional information.
Over time, multiple RAG variants have been defined.
Graph-Based RAG: In graph-based RAG, graph theory is used. A graph is defined as G = (V, E), where V is the set of nodes and E is the set of edges. In graph-based RAG, both RAG and Knowledge Graphs are used. A Knowledge Graph serves as the data structure; an edge between two nodes represents a relationship between entities. In this approach, every node has an embedding vector. Traditional RAG stops after retrieving the top K chunks.
In contrast, graph-based RAG performs graph traversal. Let R be the node. Then it expands as follows.
R1 = R0 U N(R0)
R2 = R1 U N(R1) … and so on.
Here, N denotes the neighbouring nodes. We can expand until a fixed threshold (t-hop expansion). As in other methods, traversal can use a probability transition matrix Pij (this matrix gives the probability of jumping from one node to another).
Hands-on Assignment:
Step 1: You can ask Copilot/any other tool you use to generate the script with the following prompt:
Prompt 1:
Generate a complete Python script to create a Knowledge Graph with networkx and traverse it.
Prompt 2:
Write a Python script to generate a Knowledge Graph with networkx from documents and traverse the graph. Generate demo documents.
Prompt 3:
Create a Knowledge Graph with Neo4j and traverse the graph. Generate a complete Python script.
Prompt 4:
Generate a Python script to create documents and then, from the documents, create a graph-based RAG using a Knowledge Graph data structure. Use the LangChain and LangGraph libraries. Generate the complete script and add comments to make it easy to understand.
Step 2: Run the generated scripts, then analyse the scripts and the results.
Next, we talk about planning agents. Planning agents come in two types: ReAct (Reasoning and Acting Agent) and reflection-type. To understand these, let’s first look at an AI agent’s trajectory.
h(k) = {q, t(k-1), a(k-1), o(k-1), …, t(0), a(0), o(0)}, where q is the query, t is the reasoning (what to do?), a is the action, o is the observation and k is the state/iteration.
From h, the ReAct Agent probabilistically generates the reasoning policy from the following key equations.
- Generate thought: T = argmax P(t|h)
- Generate action: A = argmax P(a|h)
- Get environment response: O = f(A)
- Repeat Step 1, Step 2, Step 3 until the solution converges.
Here, P is the conditional probability.
For the Reflection Agent, the equation changes as follows.
h(k) = {q, t(k-1), a(k-1), o(k-1), r(k-1) …, t(0), a(0), o(0), r(0)} where r is the reflection factor. It is a self-generated critique or lesson. For example, r(k-1) = “I searched the wrong source. Next time, verify the answer before responding.” The reflection factor is not a numeric reward, but it can be derived from numerical reward values.
- Reflection factor: R = g(SUM(rewards over trajectory))
The Reflection Agent can hallucinate if the reflection factor is not generated properly, since an LLM generates it.
Hands-on Assignment:
Step 1: You can ask Copilot/any other tool you use to generate the script with the following prompt:
Prompt:
Write a Python script to demonstrate a ReAct Agent with LangGraph’s built-in library. Generate the complete script and add comments so each part is easy to understand.
Step 2: Run the generated scripts, then analyse the scripts and the results.
In general, the ReAct Agent solves a task while the Reflection Agent learns from the past. In this lesson, we have done hands-on work on RAG and planning agents and learned the core mathematics. The mathematical foundation helps build a stable Agentic AI environment. In the next lessons, we will learn about some more advanced algorithms and models like Reinforcement Learning, Diffusion Models, and Large Action Models.
Leave a Reply