Guillaume Duvernay

Concrete tips to go from simple RAG to better RAG

10 min read

RAGn8nagentsAI efficiency

Almost every n8n RAG tutorial ends at the same canvas: an AI agent, and hanging off it, a vector store node used directly as a tool. I’ve built that setup plenty of times. It works, and for a while I didn’t look any closer at it.

Then I started asking it harder questions, and three things went wrong that all have the same cause. The agent owns the retrieval, and you own nothing.

This is the written version of a video, with the same biology course and the same numbers.

Video: Level Up Your n8n RAG Agents: Smart Multi-Query & Reasoning (with Supabase + GPT-5)Play

The canvas everyone starts from

My knowledge base for this is a biology course, vectorized into Supabase. Ask it “what is a cell” and it answers well. That’s the demo everyone shows, and it’s not a lie, simple questions work.

Three sub-nodes, and the retrieval is one of them. Everything that happens inside that third box is decided at runtime by the agent.

Here is what you control in that setup: a description on the tool, telling the agent when to use it. That’s the whole surface.

The left column is the entire configuration surface. Everything on the right is decided for you.

You never see the query

The agent writes the query itself and sends it. You can nudge it with the tool description and hope. On a question that needed rewording, or one where the user’s phrasing is nothing like the wording in your documents, you find out by looking at the execution log afterwards.

One query per call

A real question often isn’t one question. Ask something that needs a term defined and then that term related to another one, and a single similarity search has to cover both at once. It returns chunks that are middling for both instead of good for either.

The agent can call the tool twice, but that’s two full turns of the model, and it usually doesn’t bother.

Four chunks come back whatever you asked

This is the one that made me change the setup. The vector store returns the four closest chunks. Closest, not close.

Ask my biology course when we’re having dinner tonight, and you still get four chunks. They’re far, but they’re the four least far, so up they come. The agent receives them with no indication that they’re junk, and models are agreeable, so it treats them as relevant material.

Same shape of response as a good answer. Nothing in it tells the agent these are the best of a bad set.

Put a sub-workflow where the tool was

The fix is to stop giving the agent the vector store, and give it a sub-workflow instead. In n8n that’s the Call n8n Workflow tool: the agent calls it, a workflow runs on its own canvas, and whatever the last node outputs is what the agent receives.

The agent’s side barely changes. Same trigger, same agent, and the vector store swapped for two tools.

The agent runs on gpt-5-mini here, not a frontier model. Once the retrieval does the sorting, the agent's job is writing an answer from what it was handed, and that is not the hard part.

The sub-workflow is where all the new surface is. Every box below is a step you wrote, and one you can open in the execution log afterwards.

The filter and the IF next to it are the whole argument. One decides what is good enough, the other makes sure a query that found nothing says so instead of returning silence.

An array of queries, in one tool call

The sub-workflow’s trigger accepts a field called queries. The tool description tells the agent it’s an array of one to five, and the system prompt says the same thing in the other direction: the more complex the user’s question, the more you break it into sub-queries, up to five.

That’s the multi-query part, and it costs one tool call rather than three.

The agent decided on the breakdown by itself here. I only told it the field takes up to five.

The score is the whole point

Inside the sub-workflow, Supabase is a regular node, so I map the query myself and the response carries a similarity score. The filter drops anything at or under 0.4.

Two details that cost me time. The filter has to be set to always output data, otherwise a query where nothing passed produces no item at all and the branch after it never runs. And the IF that follows checks whether any chunk survived, because that’s the case you want to handle out loud rather than by returning silence: the agent gets a sentence saying the knowledge base had nothing useful for that query.

Tell the agent where a chunk came from

The clean-up step matters more than it sounds. Supabase returns metadata I have no use for, things like the content type of the source file, and that’s tokens on every chunk of every query.

What I keep is the chunk text plus two fields: the chapter it came from, and its relevance score rounded to two decimals. Both go to the agent. The chapter name lets it say where something comes from, and passing the score through means the agent can see that one of its five queries scraped in at 0.42 while another hit 0.81, and weigh them accordingly.

Name what you hand back

The aggregate at the end doesn’t produce an anonymous blob. It produces one field called Knowledge base retrieval, and inside it each entry pairs Query to the knowledge base with Chunks returned.

That pairing is doing real work. The agent sent three queries, it gets three labelled groups back, and it can tell which chunks answer which part of its own question. Hand it a flat array of twelve chunks instead and it has to infer the mapping, which it will do, badly, some of the time.

Naming the fields costs one Set node. It’s the cheapest thing on this list and the one I skipped for the longest.

A think step before the answer

The other change is on the agent itself. I gave it a think tool, and the system prompt tells it to use that tool right after the retrieval comes back: analyse the question against what came back, and challenge whether it actually has what it needs to answer.

The tool description caps it at 50 words. Without that it writes paragraphs, and thinking out loud about four chunks is not worth a page of tokens.

You can read those notes in the execution log, which is the part I didn’t expect to like as much as I do. When an answer comes out wrong, the notes usually tell you whether the retrieval was bad or the reasoning was.

The last lines of the system prompt are the ones I’d keep if I could keep only two. Answer only from the course content, and if a question falls outside it, redirect rather than answer. And if you don’t have what you need, say so rather than answering from general knowledge anyway.

Both only work because the retrieval below them is honest about coming back empty. An instruction to admit ignorance is worthless when every query returns four chunks that look like evidence.

What this is really teaching

Every fix here is the same fix. Something was being decided for me, I moved it into a step I own.

That’s worth more than the workflow. When I work in Claude Code now, the questions I ask are the ones this setup forced me to ask: what exactly got searched, how much came back, how much of it was worth reading, what is sitting in the context window that nobody needed. A generic agent hides all of that by default, and it will happily read twenty files to answer something that needed two.

Building the retrieval by hand once is how you learn to notice.

Sources