Open WebUI Knowledge Base: Setup and Configuration

A Knowledge Base in Open WebUI is a retrieval-augmented generation (RAG) system that lets you upload documents and search them in real time. Unlike Notes, which inject full content into every message, Knowledge Bases retrieve only relevant passages on demand. This means you can maintain massive document libraries without overwhelming your model's context window.

What is Open WebUI Knowledge Base? (And Why It Beats Notes)

Open WebUI's Knowledge Base is a RAG system powered by ChromaDB (or other vector databases like Qdrant). It automatically chunks documents, embeds them, and stores vectors for semantic search. When you query, it retrieves the most relevant passages instead of injecting entire files. This approach scales to thousands of pages without bloating your prompt.

Notes and Knowledge Bases serve different purposes. Notes inject the full content into every message your model receives. They work well for small reference documents--maybe a 2-3 page product spec or standard operating procedure. Your model sees everything, so nothing gets missed. But if you upload a 500-page manual or a folder of research papers, Notes will consume 90% of your context window before your model answers a single question.

Knowledge Bases solve this by finding relevant chunks first. You ask a question, the system searches your entire library (no matter how large), and returns only the 10-20 most relevant passages. Your model answers based on those passages, not the entire dataset. The tradeoff: if the relevance ranking misses something important, your model won't know it exists. This is why RAG configuration matters so much.

When to use each: Notes for small, critical reference docs that must always be in scope. Knowledge Bases for team documentation, research libraries, compliance archives, or any collection larger than a few pages. Open WebUI's GitHub repository has 158.1K stars and 23K forks, driven partly by community demand for exactly this RAG capability.

Prerequisites: What You Need Before Starting

Open WebUI requires a running instance with admin access to the Settings panel. If you run Open WebUI locally via Ollama, you already have everything you need. If you're deploying at scale for a team, managed hosting removes the infrastructure burden.

You need at least one language model capable of understanding search results and generating answers. This can be an Ollama model like Mistral 7B or Qwen 2.5 14B, or an API model like OpenAI's GPT-4. For document chunking and embedding, Open WebUI provides default models like nomic-embed-text, which runs locally. If you prefer cloud embeddings, you can configure alternatives.

Hardware-wise: a machine with at least 8 GB RAM runs Open WebUI and a small model. For team deployments with multiple concurrent users, 16-32 GB is more practical. Your vector database (ChromaDB) stores embeddings in memory, so more documents mean more RAM. For 10,000+ documents, consider cloud storage for ChromaDB or switch to a managed Qdrant instance.

Admin access to the Settings panel is required to configure global document settings, embedding models, reranking, and limits. Non-admin users can create and use Knowledge Bases, but they can't change system-wide settings.

Step 1: Create Your Knowledge Base

In Open WebUI, navigate to Workspace > Knowledge to create a new Knowledge Base. Click the plus button, name it (e.g., "Team Policies," "Research Library"), and optionally add a description. The name is for your reference; the description appears to team members.

Open WebUI accepts PDF, Word documents (.docx), Markdown, plain text, and HTML. Upload a single file, multiple files at once, or an entire directory. The system queues files asynchronously, so uploads happen in the background. Check the status indicator next to each file to see when processing is complete.

For large uploads, use the oikb CLI tool (open-webui-kb) instead of the browser. oikb supports incremental syncing, parallel uploads, and scheduled jobs. It can also pull directly from Git repositories, Confluence, S3 buckets, or local directories. If you're syncing a 500-file folder weekly, oikb saves hours of manual work.

After uploading, documents are chunked automatically. By default, Open WebUI uses 1500-token chunks with 200 tokens of overlap. This keeps sentences intact and improves search coherence. The text is then embedded using your configured embedding model and stored in the vector database.

Step 2: Configure RAG Settings (The Critical Part)

RAG configuration determines retrieval quality. Poor settings mean your Knowledge Base returns irrelevant or incomplete results. Good settings mean fast, accurate answers. Open these settings in Admin Panel > Documents.

The embedding model is your first choice. nomic-embed-text is the default and recommended for most setups. It's free, runs locally on Ollama, and produces high-quality embeddings. If you prefer cloud embeddings, you can configure Mistral Embed or OpenAI's text-embedding-3-small. Changing embedding models requires re-indexing your documents.

Chunk size controls how the system breaks documents into searchable pieces. The default 1500 tokens keeps most paragraphs intact. Larger chunks (2000+) give more context per result, but miss fine-grained relevance. Smaller chunks (1000 or less) improve precision but may split important concepts. Chunk overlap (default 200 tokens) keeps repeated sentences at chunk boundaries so searches find them even at chunk edges.

Hybrid search combines two retrieval methods: semantic search (vector similarity) and lexical search (exact keyword matching with BM25). Enable both. Semantic search finds conceptually related content even if wording differs. Lexical search catches exact phrases and technical terms that vectors might miss. Together, they're more robust than either alone.

The reranker model ranks results by relevance after retrieval. bge-reranker-v2-m3 is recommended. It runs locally and re-orders the top 100 candidates by confidence, moving the most relevant to the top. This step dramatically improves answer quality without slowing performance. Enable it.

Top K is the number of chunks your model receives per search. Default is 10. For shorter answers, 10-15 is usually enough. For complex questions requiring multiple sources, use 20-30. Higher Top K means more context but risks confusing the model with contradictory information. Tune this based on your model's context window and the typical complexity of questions.

Here is a configuration cheat sheet:

Step 3: Attach Knowledge to a Model or Chat

Knowledge Bases aren't automatically used by your models. You must explicitly attach them. There are two approaches: attach to a custom model, or reference in-chat.

To create a custom model that always uses a specific Knowledge Base, go to Admin Panel > Models. Click Create, give it a name like "Mistral+Support Docs," and under Knowledge Base, select which ones to attach. When anyone uses this model, it automatically searches attached KBs. You can attach multiple KBs to one model.

Alternatively, in any chat, type #KnowledgeBaseName to reference a KB inline for just that conversation. This is useful for one-off research or testing.

The critical detail: your model must support native function calling. Most modern models (Mistral, Qwen, Llama 2 70B, GPT-4) do. Older or smaller models may not. If your model doesn't support native function calling, add a system prompt that instructs it to ask the user questions if it needs information. You can also manually pass retrieval results to the model via the API.

Here is an example system prompt for a model with attached Knowledge Bases:

You are a helpful support assistant. Before answering questions, search the attached Knowledge Base for relevant documentation. If you find relevant passages, cite them in your answer. If you don't find anything, tell the user the answer isn't in your documentation and suggest they contact support.

This prompt ensures the model actually uses the Knowledge Base instead of relying on its training data.

Step 4: Share Your Knowledge Base with Your Team

Share a Knowledge Base via the Access menu on the KB row. Options include: Public (anyone with access to your Open WebUI instance can see it), Private (only you), or assigned to specific users or groups.

Permissions are set per Knowledge Base. You can't share a subset of documents within a KB; it's all-or-nothing. If you need different teams to see different documents, create separate Knowledge Bases.

Group-based sharing is powerful for larger teams. Create a group in Settings > Users (if you have admin access), add team members to it, then share KBs with the group. This scales permissions management without updating each user individually.

Non-owners cannot delete documents from a shared KB, but they can create new Knowledge Bases. Each user can create their own private KBs or contribute to shared ones. There's no approval workflow; once shared, a KB is immediately usable.

Multi-team setups work best with separate KBs. For example: HR team gets "Employee Handbook," Engineering gets "Technical Docs," Sales gets "Product Specs." Each team attaches their KB to their own custom model or references it in chat. This keeps information organized and prevents accidental misuse (e.g., sales reps accidentally answering HR questions with product knowledge).

Understanding Knowledge Base Limits and Scaling

Admin can set file size and document count limits in Admin Settings > Documents. Configurable limits include: maximum file size per upload (e.g., 100 MB), maximum files per upload batch, and maximum files per Knowledge Base. These defaults prevent someone from accidentally uploading an entire hard drive.

Per-Knowledge Base overrides let you set stricter limits on individual KBs. For example, create a public KB with a 5-file limit so users don't fill it with personal documents.

For large deployments (10,000+ documents), performance depends on vector database choice. ChromaDB (default, in-memory) is fast for small collections but consumes RAM linearly. Switching to Qdrant or PGVector (persistent, scalable) improves performance for large KBs. This is an admin configuration change.

Directory organization doesn't affect search; all files in a KB are searchable regardless of folder structure. But directories help users navigate. You can drag-and-drop to reorder folders and files for clarity.

For bulk uploads and syncing, the oikb CLI tool is indispensable. It handles incremental updates (only new or changed files are re-indexed), parallel processing (faster than single-threaded uploads), and scheduling (sync every night at 2 AM). oikb can pull from Git repos, Confluence wikis, or S3 buckets, making it ideal for documentation teams that version control their docs.

Troubleshooting: Why Your Knowledge Base Isn't Working

1. Model doesn't search the Knowledge Base. Check that the model has native function calling enabled and that the KB is attached. Try sending a test query. If results are empty, verify that the KB has processed files (check status next to each file name). If files are still processing, wait or check logs for indexing errors.

2. Results are thin, incomplete, or irrelevant. Increase Top K (retrieve more chunks). Enable hybrid search if disabled. Try adjusting chunk size: smaller chunks (1000) improve precision, larger chunks (2000) improve context. Check reranker is enabled. If the KB is new, wait a few minutes for indexing to complete fully.

3. A document I uploaded isn't being found. Check that the file format is supported (PDF, docx, md, txt, html). If it's a PDF with images only (no text), it may not extract content without an OCR engine (Tika or Docling). Check admin logs for extraction errors. Try re-uploading or converting the file to a text-friendly format first.

4. Permission errors when accessing a shared KB. Verify the KB is actually shared with you (check Access settings). If shared with a group, confirm you're in that group. If ownership changed, ask the new owner to re-share. Refresh your browser; permissions cache for a few minutes.

5. Slow retrieval times. Profile the bottleneck: is indexing slow or search slow? Check vector database (ChromaDB vs. Qdrant). If you have 100,000+ embeddings in ChromaDB, consider migrating to Qdrant. Reduce Top K if possible. Check your embedding model performance (local Ollama models are slower than cloud APIs).

6. API uploads time out. The Knowledge Base API returns immediately after queuing, not after completion. Poll the status endpoint before querying. Async processing is by design to avoid timeouts. Check admin logs for processing errors if status stays pending for hours.

Frequently Asked Questions

What is the difference between Knowledge Base and Notes in Open WebUI?

Notes inject full content into every message, suitable for small reference docs. Knowledge Bases use RAG to retrieve only relevant passages, scaling to thousands of pages without overwhelming context. Choose Notes for small files, Knowledge Bases for large libraries or frequent lookups.

How do I configure an embedding model for my Knowledge Base?

Go to Admin Panel > Documents and select your embedding model under "Embedding Model." nomic-embed-text (local) is recommended. Changing models requires re-indexing all documents. Cloud alternatives like Mistral Embed or OpenAI embeddings are available if you prefer.

What are the file size and document count limits?

Admins set configurable limits in Admin Settings > Documents (e.g., 100 MB per file, 1000 files per Knowledge Base). Per-KB overrides can enforce stricter limits. There is no hardcoded ceiling; limits are a protection against accidental bulk uploads.

How do I share a Knowledge Base with team members?

Click the Access menu on the KB row. Select Public (anyone can access) or assign to specific users or groups. Groups make permission management easier for larger teams. Sharing is instant; no approval workflow.

What are the best RAG settings for optimal retrieval?

Enable hybrid search (semantic + keyword), use bge-reranker-v2-m3, set Top K to 10-20, and chunk size to 1500 tokens. These defaults work for most use cases. Tune Top K based on your model's context window and answer complexity.

Can I use multiple Knowledge Bases in one chat?

Yes. Create a custom model with multiple KBs attached, or reference multiple KBs inline using the #KBName syntax. Queries search all attached KBs and return consolidated results.

Why isn't my Knowledge Base being used by my model?

Verify the model supports native function calling. Ensure the KB is attached to the model (or referenced in chat with #KBName). Add a system prompt instructing the model to search. Check that files in the KB are fully indexed (status shows complete).

The Bottom Line

Open WebUI's Knowledge Base turns a local AI into a team research tool. RAG retrieval means you can upload documentation freely without worrying about context limits. Configuration matters: enable hybrid search, reranking, and tune Top K for your use case. Sharing KBs across teams is straightforward, and for large uploads, the oikb CLI tool scales easily.

If you're managing Open WebUI for a team, consider hosted infrastructure to eliminate server management overhead. Opsily's managed Open WebUI takes care of updates, backups, and scaling so you focus on Knowledge Base content. Ready to deploy at scale? Get started with managed Open WebUI hosting.