AI & LLM Tools

How to Build RAG in n8n: Complete Setup Guide

J
James Eriksson
··18 min read
Learn to build a RAG pipeline in n8n without code. Complete walkthrough: document ingestion, vector storage, retrieval agents, and production deployment.
TL;DR
  • RAG prevents LLM hallucinations by feeding the model real, retrieved data before answering questions.
  • n8n's visual editor lets you build complete RAG pipelines without coding: document loading, chunking, embedding, vector storage, and retrieval.
  • Separate your ingestion workflow (runs once daily) from your query workflow (runs per request) to avoid performance bottlenecks.
  • Choose Pinecone for simplicity, Qdrant for cost at scale, or Supabase pgvector if you already use Postgres.
  • Managed n8n ensures availability and compliance for production; self-hosted works for internal tools and prototypes.

RAG, or Retrieval Augmented Generation, is a simple pattern: give your AI model real data before it answers a question. This prevents the hallucinations that plague generic large language models. n8n is uniquely positioned to build RAG pipelines because its visual workflow editor handles the entire pipeline--document ingestion, chunking, embedding, vector storage, retrieval, and LLM integration--without writing a line of code.

What is RAG and Why n8n?

RAG retrieves relevant documents from a database before passing them to an LLM, grounding the model's response in verified data. n8n excels at this because its visual node-based interface lets you wire ingestion, retrieval, and LLM calls without code, making it fast to prototype and simple to modify.

The core problem RAG solves is straightforward. Large language models learn from training data with a knowledge cutoff. They cannot access your internal documents, proprietary databases, or real-time information. When prompted with a question outside their training data, they invent answers--a behavior called hallucination. RAG intercepts the query, retrieves the most relevant documents from your knowledge base, and feeds those documents to the LLM along with the question. The model then answers based on the retrieved context, not its raw training. This drops hallucination rates dramatically and lets you trust the model's response.

Why n8n for RAG? Because RAG is a workflow problem, not a coding problem. You need to load documents from somewhere (Google Drive, S3, your database), split them into chunks, convert chunks to embeddings, store embeddings in a vector database, retrieve similar embeddings when a question arrives, and pass the results to an LLM. That is five discrete steps, each with decision points (which embedding model, which vector DB, which LLM). n8n's node-based editor maps these steps directly onto visual blocks. You connect a Document Loader node to a Text Splitter node to an Embedding node to a Vector Store Write node. No YAML. No Python. Just wiring.

n8n has momentum behind it. The platform raised $240 million in Series C funding in October 2025 at a $2.5 billion valuation. The codebase has 210.6 thousand stars on GitHub. The platform offers 1,500+ integrations, so connecting RAG to your existing tools--Slack, Discord, Google Workspace, your CRM--is drag-and-drop work. For a 10- to 100-person company building a prototype RAG system or a production workflow, n8n removes friction.

The Complete RAG Architecture in n8n

RAG in n8n uses two separate workflows: an ingestion workflow that processes documents into a vector database once, and a query workflow that runs each time a user asks a question. Separation is critical: ingestion is slow and expensive; querying must be fast and cheap. Running both in one workflow breaks performance.

The ingestion workflow runs on a schedule or triggered manually. It fetches documents from a source (a folder of PDFs, a Notion database, a web crawler output), chunks them into overlapping segments, generates embeddings for each chunk, and writes the embeddings plus metadata (document name, source URL, creation date) into a vector database. This workflow runs once per day or once per update cycle. Ingestion is resource-intensive because embedding thousands of chunks costs API calls (if you use OpenAI) or GPU time (if you use a local model like Ollama). You want this to be predictable and scheduled.

The query workflow is the opposite: it needs to be fast and always available. A user (or a webhook, Slack bot, API caller) submits a question. The query workflow: (1) converts the question to an embedding using the same embedding model as ingestion, (2) searches the vector database for the top K most similar embeddings, (3) retrieves the original chunks associated with those embeddings, (4) passes the chunks plus the original question to an LLM, and (5) returns the LLM's answer. This entire cycle should complete in 2-5 seconds. Running ingestion and querying in parallel on the same n8n instance starves the query workflow of resources.

The separation is the key architectural insight. Many teams try to build a single-workflow RAG and hit a wall: slow queries, timeouts, dropped requests during document processing. Splitting into two workflows means you can tune ingestion for cost and accuracy separately from query performance. You can also scale them independently: assign query workflows to a faster, always-on execution environment and let ingestion run on a slower, batch environment. This is where Opsily's managed n8n platform shines: you can set separate execution policies for each workflow without managing the infrastructure yourself.

Building the Ingestion Pipeline

The ingestion pipeline loads documents, splits them into chunks, converts chunks to embeddings, and writes them to a vector database with metadata. Each step is a node in n8n: Document Loader (or HTTP Request to Google Drive), Text Splitter, Embedding node, Vector Store Write. Configuration choices at each step affect accuracy and cost.

Step 1: Load documents. In n8n, you use a Document Loader node (if your source is directly supported: PDF, TXT, Markdown) or an HTTP Request node to fetch documents from an API. If your documents live in Google Drive, you can use the Google Drive node to list files, then HTTP Request to fetch each file's content. If you use Notion, the Notion node retrieves page content. If documents are on the web, use the HTTP Request node or a web scraper node. The goal here is simple: get document content as text into n8n.

Step 2: Chunk the text. Documents are too long to embed as a whole (embedding APIs have token limits; most max out around 8,000 tokens per request). The Text Splitter node breaks documents into smaller pieces--typically 500 to 2,000 tokens per chunk, with 100 to 500 tokens of overlap. Overlap matters: if a keyword appears near the boundary between two chunks, overlap ensures it is captured in both, so retrieval finds the answer even if the question is phrased ambiguously. The splitter also preserves metadata (which document the chunk came from, its original position).

Step 3: Generate embeddings. The Embedding node calls an embedding API (OpenAI's text-embedding-3-small costs $0.02 per 1 million input tokens; Cohere Embed costs $0.10 per 1 million tokens; or you run Ollama locally for free). The embedding converts each chunk into a 768- or 1,536-dimensional vector. These vectors are what the vector database indexes and searches. Choice here is important: if you embed with OpenAI and later switch to Ollama, your vectors will not be compatible (different dimensions, different math), and you must re-embed everything.

Step 4: Write to vector database. The Vector Store Write node takes the chunk, embedding, and metadata, and pushes them into your vector database (Pinecone, Qdrant, Weaviate, or Supabase pgvector). The vector database indexes these vectors so that later, when a query arrives, it can quickly find the K most similar embeddings. Most vector databases support metadata filtering: you can tag a chunk with 'document_type: contract' and later filter retrieval to only contracts.

A complete ingestion workflow might look like: Google Drive node to HTTP Request (fetch file content) to Text Splitter (chunk at 1,000 tokens, 200 overlap) to Embedding node (call OpenAI text-embedding-3-small) to Vector Store Write (write to Pinecone). Then set this workflow on a Daily trigger at 2 AM. When it runs, it fetches new documents from your Drive, chunks and embeds them, and updates the vector database. Cost for embedding 10,000 chunks at $0.02 per million tokens: roughly $0.20 per ingestion run if each chunk averages 100 tokens.

Vector Store Options and How to Choose

Pinecone (managed, $0.04 per million queries), Qdrant (self-hosted or managed, cheaper at scale), Weaviate (open-source, self-hosted or managed), and Supabase pgvector (Postgres native, cheapest for small volumes). Choose based on: volume of documents, query frequency, budget, and whether you want managed or self-hosted.

Pinecone is the easiest entry point if you are starting small and want zero operations overhead. Pinecone is a fully managed vector database: you create an index, send documents, and query. Pricing is $0.04 per 1 million queries plus storage. For a company with 1 million vectors stored and 1,000 queries per day (365,000 per year), the cost is roughly $0.015 per month in queries plus a bit more in storage. Pinecone handles scaling, backups, and availability for you. Downside: you are locked into Pinecone's API; data export is possible but cumbersome. Also, for very small scales (under 10,000 vectors, under 100 queries per day), Pinecone may feel expensive compared to running Qdrant locally.

Qdrant is the open-source alternative and the choice of teams that want control or lower cost at scale. Qdrant runs as a Docker container on your own server (or on managed Qdrant cloud). Self-hosted Qdrant costs zero if you already have infrastructure; managed Qdrant cloud starts at $0 for a hobby tier (1 GB storage) and scales to pay-as-you-go. Qdrant is efficient: it handles large collections and supports hybrid search (keyword plus semantic). Downside: you own operations (backups, upgrades, availability). If self-hosted, it requires server resources (2-4 GB RAM for millions of vectors, more for GPU acceleration).

Weaviate is similar to Qdrant: open-source, self-hosted or managed cloud. Weaviate emphasizes multimodal support (images, audio, text in the same database) and generative search (built-in LLM integration for result re-ranking). For text-only RAG, Qdrant is simpler. For multimodal or if you want the LLM tightly integrated into retrieval, Weaviate is interesting.

Supabase pgvector is the outlier here: it is not a specialized vector database but Postgres with a pgvector extension. If you already use Postgres for your application, adding pgvector to the same database is tempting: one database to manage, one connection string, one backup strategy. Supabase (the managed Postgres host) charges $25 per month for a small instance (which handles millions of vectors). For a new company not yet on Postgres, Supabase is not cheaper than Pinecone for low volumes. But if you are on Postgres already, it is elegant.

How to choose: If this is your first RAG project and you have a budget, use Pinecone. It is managed, reliable, and you learn RAG without wrestling ops. If you want to avoid vendor lock-in, use Qdrant self-hosted. If you are already on Postgres or want a cheap self-hosted option, pgvector in Supabase or your own Postgres is solid. If you want the latest features and multimodal support, Weaviate is worth testing.

Building the Query Workflow

The query workflow listens for a question (via webhook, Slack, HTTP API), retrieves similar chunks from the vector database, passes them to an LLM with the original question, and returns the answer. It is simple: Trigger to Embedding node to Vector Store Query to LLM node to Respond.

A query workflow starts with a Trigger node. This could be an HTTP Request trigger (so external apps can call it via API), a Slack trigger (react to @mention in Slack), a Webhook trigger, or a Schedule trigger (run every minute to check for new questions). For a customer-facing chatbot, HTTP Request is common: your frontend calls the n8n workflow API with the user's question.

Next, the Embedding node converts the question to a vector using the same embedding model as ingestion. This is critical: if ingestion uses OpenAI, the query must also use OpenAI. If ingestion uses Ollama, query must use Ollama. Mismatched embeddings will fail.

The Vector Store Query node searches the vector database for the top K most similar embeddings (usually K=5 to K=10). It returns the original chunks, sorted by similarity score. You can optionally add metadata filtering here: 'only return chunks from documents updated in the last 30 days' or 'only search the contracts folder'. Metadata filtering is powerful because it lets you scope RAG to a specific subset of your knowledge base.

The LLM node is where the magic happens. It takes the retrieved chunks, the original question, and a system prompt, and generates an answer. A typical prompt looks like:

"You are a helpful assistant. Use the following context to answer the question. If you cannot answer based on the context, say so.

Context: [Retrieved chunks]

Question: [User question]

Answer: [LLM generates response]"

You can use any LLM: OpenAI GPT-4 ($0.03 per 1,000 tokens), Claude, Ollama (free, local), or others. For latency-sensitive applications (sub-second response needed), Ollama is better; for accuracy, GPT-4 or Claude are stronger.

The response is returned to the trigger. If it was a Slack trigger, the answer posts to Slack. If it was an HTTP Request, the answer is returned in the HTTP response. If you want to log the conversation for debugging or compliance, add a node to write the Q&A pair to a database or file storage.

Memory (conversation history) is optional. If your RAG is just question-answer pairs with no back-and-forth, you don't need memory. If you want multi-turn conversations or more complex logic like deciding when to retrieve additional context, you'll need stateful handling--these advanced patterns are explored in our guide on building n8n AI agents.

Performance and Scaling: Self-Hosted vs. Managed

Self-hosted n8n on your server or cloud instance gives you full control and lower cost up to about 1,000 workflows per day. Managed n8n via Opsily or n8n Cloud handles autoscaling, failover, and compliance, starting around 10,000 workflows per day. For production RAG, managed is safer; for experiments, self-hosted is cheaper.

Self-hosted n8n runs as a Docker container on any server: your laptop, an AWS EC2 instance, a DigitalOcean droplet, or your own bare metal. The minimum spec is light: 2 GB RAM, 1 CPU, and 5 GB disk for the application code. If you run Ollama (local LLM) alongside n8n for RAG queries, you need a GPU for acceptable speed. An RTX 3060 (12 GB memory) handles 7-10 queries per minute; an RTX 4090 handles 50+. Total cost: $300 for an RTX 3060 GPU, or $60 per month for a GPU-enabled cloud instance. Self-hosted works well for internal tools, moderate volume (under 1,000 queries per day), and when you control the infrastructure. Our guide to self-hosted AI with RAG covers hardware profiles and local model setup in detail.

Managed n8n (like Opsily's hosted platform or n8n Cloud) handles scaling for you. When a query spike hits, the platform adds execution capacity automatically. You don't wake up to a PagerDuty alert because a query workflow maxed out. Managed n8n also includes deployment automation, built-in monitoring, and GDPR compliance (data residency in EU, encryption, audit logs). For a SaaS product exposed to customers, managed is the smarter choice. Cost starts around $29 per month for a hobby plan and scales to enterprise pricing for high volume.

For RAG specifically: your vector database (Pinecone, Qdrant) is independent of n8n, so the n8n choice only affects the retrieval and LLM call orchestration. If Pinecone is down, n8n cannot save you; if your LLM provider is down, n8n waits. But if n8n itself is down, your entire RAG pipeline halts. Using managed n8n ensures RAG availability.

If you are building a prototype for internal use (employees asking questions about company docs), self-hosted is fine. If you are building a customer-facing product (SaaS, user-generated queries), managed gives you the operational security and scaling you need without hiring a DevOps engineer.

Best Practices and Common Pitfalls

Chunk size strongly affects accuracy: too small, you lose context; too large, you waste embedding cost. Reranking (re-scoring retrieved results) filters noise. Monitor vector similarity scores and LLM tokens to catch degradation. Common failures: metadata field mismatches, embedding model swaps, and vector database syncing.

Chunk sizing is not obvious. Too small (100 tokens), and a question might need context spread across multiple chunks; the LLM has to infer connections. Too large (3,000 tokens), and you pay more for embedding, slower retrieval, and you sometimes surface irrelevant context. The sweet spot for most RAG systems is 500-1,500 tokens per chunk with 200-token overlap. Test on your real documents: chunk a representative document set, embed them, and manually score retrieved results on a handful of test questions. Adjust chunk size until retrieval quality stabilizes.

Reranking is a second-stage filter applied after vector search. The vector database returns the top K=20 results by similarity score. A reranking model (a smaller, faster ML model) re-scores those 20 and surfaces the top 5 to the LLM. This is expensive (adds latency) but critical if your first-stage retrieval is noisy. Tools like Cohere Rerank or LangChain's reranking support help. For small, well-curated knowledge bases, reranking is not needed; for large noisy corpuses, it cuts hallucination rates.

Monitor embedding similarity scores. When a query arrives and the top retrieved chunk has a similarity score of 0.6 or lower, you are in the risk zone: the LLM is being asked to answer from barely-related context. Log this. If more than 10% of queries have low similarity, your chunk size or embedding model may be wrong.

Monitor token costs. Each LLM call costs tokens (input tokens from the question and context, output tokens from the answer). If your query workflow suddenly uses 5x the tokens it used last week, something broke. Either your retrieval is surfacing huge chunks, or your system prompt is bloated. Catch this early before the bill surprises you.

Common failure: metadata mismatch. You set a field 'document_id' during ingestion but try to filter by 'doc_id' during query. The vector store silently ignores the filter, and you get wrong results. Use consistent naming. Write a test query that checks metadata filtering works as expected.

Common failure: embedding model swap. You ingest documents with OpenAI Embedding, then switch to Cohere during queries because Cohere is cheaper. The query embeddings are in Cohere's space; the stored embeddings are in OpenAI's space. They don't match. Result: retrieval returns random chunks. If you must switch embedding models, you must re-embed all stored vectors. This is expensive. Choose an embedding model and stick with it or plan for a full re-embedding when you switch.

Common failure: vector database sync lag. You update documents in your knowledge base, re-ingest, and expect the vector database to be updated immediately. But the ingestion workflow is still running. A query in the middle of ingestion returns stale results. Solution: have a manual trigger for urgent updates or a priority queue system that ingests new docs in minutes, not hours.

Frequently Asked Questions

What is the difference between Naive RAG and Agentic RAG?

Naive RAG retrieves documents once and passes them to the LLM. It is fast, deterministic, and cheap but struggles with vague questions and cannot self-correct. Agentic RAG lets the LLM decide whether to retrieve more, reorder results, or use tools. It handles complex queries but is slower and costs more in tokens. n8n supports both: naive RAG is the simple query workflow described here; agentic RAG requires adding a loop that lets the LLM agent decide when to retrieve again.

What are the best vector databases to use with n8n?

Pinecone (managed, $0.04 per 1 million queries), Qdrant (self-hosted, free; managed cloud, starting free), Weaviate (open-source, self-hosted or managed), and Supabase pgvector (Postgres native, $25 per month for managed). Pinecone is easiest for beginners. Qdrant and Weaviate are cheaper at scale. Supabase pgvector is best if you already use Postgres.

How does RAG differ from fine-tuning?

RAG adds external knowledge at query time without retraining the model. It is fast to update (new docs take hours), cheap (you don't retrain), and flexible (you can change your knowledge base per query). Fine-tuning bakes knowledge into model weights via retraining. It is slower to update (retraining takes days), more expensive, but can achieve higher accuracy on very specific tasks. For most use cases, RAG is simpler.

Should I use n8n or a dedicated RAG framework like LangGraph?

n8n is visual and needs no coding; it is good for single-agent workflows and teams that want to avoid Python. LangGraph (from LangChain) is code-first; it gives you fine-grained control over agentic logic, multi-agent orchestration, and custom reasoning loops. Use n8n if you want speed to prototype and simplicity for internal tools. Use LangGraph if you need complex multi-agent systems or your team codes in Python already.

What is the self-hosted AI Starter Kit?

A Docker Compose template that bundles n8n plus Ollama (local LLM), Qdrant (vector database), and PostgreSQL (logs and memory). It supports CPU-only, NVIDIA GPU, and AMD GPU profiles. It is free and lets you run the entire RAG stack on your laptop or a single server. Great for learning or proof-of-concept; not suitable for production at scale.

How do I monitor costs and token usage in n8n RAG?

n8n has limited built-in monitoring. You must manually log LLM API calls and embedding calls to a database or file, then aggregate. Track tokens spent, queries per second, and embedding API cost per day. Use third-party observability tools like LangFuse to log and analyze RAG chains at scale. For self-hosted plus Ollama RAG, costs are low (just infrastructure); for cloud APIs (OpenAI, Cohere), costs can surprise you if not monitored.

Can I use n8n RAG without a GPU?

Yes. Use cloud-based embeddings (OpenAI, Cohere) and cloud-based LLMs (OpenAI, Claude, Anthropic). Your n8n instance itself needs minimal compute (2 GB RAM, 1 CPU). Vector database queries are fast on CPU. The only CPU bottleneck is if you use a local LLM (Ollama) for queries without a GPU; then you get 1-2 queries per second instead of 10+. For production, CPU plus cloud models is normal.

The Bottom Line

RAG prevents LLM hallucination by grounding answers in real data. n8n makes RAG accessible: visual workflows, 1,500+ integrations, no coding required. The architecture is simple: separate ingestion and query workflows, choose a vector database (Pinecone or self-hosted Qdrant), and wire LLM calls. Start with self-hosted for experiments; move to managed n8n for production. The cost of RAG is dominated by embedding and LLM API calls, not infrastructure.

Building RAG on n8n takes a week for a prototype and a month for production, depending on your data volume and accuracy requirements.

Get started by exploring Opsily's managed n8n platform, which handles scaling, compliance, and availability for you so you can focus on your RAG logic.

Run RAG on Managed n8n
Opsily's managed n8n platform scales your RAG workflows automatically with GDPR compliance, built-in monitoring, and zero operations overhead.
Get Started Free

Ready to self-host your own apps?

One server. Multiple apps. No per-app fees.

Get started →