Open WebUI Agents: Setup Guide & How They Work

Open WebUI agents are models that can execute tasks autonomously by calling tools, running terminal commands, searching the web, or accessing knowledge bases. They differ from simple chat models because they reason through multi-step workflows and can handle errors without your intervention. This guide walks you through setting up agents, choosing the right framework, and deploying them safely.

What Are Open WebUI Agents?

An agent in Open WebUI is a language model given the ability to use external tools and execute code. When you ask an agent a question, it decides what tool to call, interprets the result, and either returns an answer or calls another tool. That loop continues until it solves the problem.

Think of it like this: a regular chat model is a person answering questions from memory. An agent is that same person, but with access to a phone, a file cabinet, and a terminal. They can look up information, write and execute code, or trigger workflows.

The key difference is autonomy. You do not need to manually call each tool. The agent figures out the chain of actions. This matters for workflows like "find all orders from last month, calculate total revenue, and email it to accounting." A chat model cannot do that without you breaking it into steps. An agent can.

Open WebUI v0.13.0 (released Nov 4, 2026) introduced the Open WebUI Computer agent, which can control your desktop or browser like a human would. That is new. Before that, agents relied on tool calling, code execution, and external API integrations.

Three Paths to Adding Agent Capabilities

You have three ways to give a model agent capabilities in Open WebUI: external agent frameworks, custom agents with tools, or a mix of both.

External agent frameworks come pre-built with specific strengths. You connect them via API and they handle tool calling internally. Examples are Hermes Agent (from Nous Research), OpenClaw (open-source shell access), and the new Open WebUI Computer.

Custom agents start with a base model, add a system prompt that tells it how to behave, and give it access to tools or a terminal. You build the tool set yourself. This gives you full control but requires more setup.

Hybrid approaches combine both. For example, use Open WebUI Computer for visual tasks and a custom agent with code execution for data processing. Different jobs use different agents.

Each has tradeoffs. External agents are faster to start but less flexible. Custom agents take longer to configure but fit your exact workflow. Hybrid setups are powerful but complex to manage.

External Agent Frameworks: Hermes, OpenClaw, and Open WebUI Computer

Open WebUI ships with three pre-built external agent options. They integrate via API and handle their own tool-calling logic.

Hermes Agent, built by Nous Research, is specialized for tool use and reasoning. It works with OpenAI-compatible endpoints and includes built-in tools for web search, code execution, and file access. Setup is straightforward: connect your model endpoint, enable Hermes in the agent settings, and start asking multi-step questions. The main limitation is that it requires a capable base model--older or smaller models fail at reasoning.

OpenClaw is an open-source agent focused on shell access and command execution. It is lighter than Hermes and better for infrastructure tasks like deployment, file management, or system monitoring. If your workflow involves running scripts or terminal commands, OpenClaw is usually the better choice. It can also handle tool calling but excels at shell integration.

Open WebUI Computer is the newest addition in v0.13.0. It can control your desktop, open applications, click buttons, type text, and navigate browsers. This is fundamentally different from the other two because it operates at the UI level, not the API level. For tasks like "scrape this website" or "fill out this form," it is uniquely capable. The tradeoff is speed: clicking and waiting for UI elements takes longer than API calls. Use it when you need to automate tasks that do not have APIs.

To choose between them: use Hermes for general reasoning and multi-step workflows, OpenClaw for infrastructure and command-line tasks, and Open WebUI Computer for GUI automation.

Each external agent is enabled in the Admin panel under Integrations. You provide an endpoint URL (if needed) and authentication credentials, then select the agent type when creating a new chat.

Custom Agents: Tools, Functions, and Pipelines

If external agents do not fit your exact needs, build a custom agent. This means choosing a base model, writing a system prompt, and defining the tools it can call.

Base Model Selection matters more for agents than for chat. Older and smaller models cannot reason well enough to decide when and how to use tools. Llama 3.1 7B and larger models work reliably. Claude and Mistral also work well. Avoid older GPT-3.5 models or tiny open-source models--they frequently call the wrong tool or get stuck in loops.

System Prompt is the blueprint for agent behavior. It tells the model what role it plays, what tools are available, and how to think. A good system prompt for a coding agent might be: "You are an expert Python engineer. You can read and write files, run tests, and commit to Git. Break tasks into small steps and always verify output. If a test fails, debug before moving on."

The prompt shapes the model's reasoning. A vague prompt leads to ineffective tool use. A detailed one with examples and constraints works much better.

Tools and Functions in Open WebUI are the actual capabilities. You can expose:

Built-in tools like web search, file access, and code execution
Python functions you write (for custom logic)
External APIs via OpenAPI specs
Terminal access (via Open Terminal Docker container)

Each tool needs a name, description, and parameters. The agent reads these descriptions to decide which tool to use.

Pipelines chain tools and models together. Open WebUI supports N8N workflows and LangGraph integration. For example, a pipeline might be: (1) agent decides to fetch data, (2) calls a web search tool, (3) parses results with Python, (4) stores in a database, (5) returns summary to user. Pipelines let you build complex workflows without writing code--you drag and drop in the UI.

Memory is also part of custom setup. Enable conversation memory so the agent remembers context across messages, or use knowledge bases (RAG) to give it access to documents without retraining. This is crucial for multi-turn workflows where the agent needs to remember earlier decisions.

Extending Agents with MCP (Model Context Protocol)

MCP is a protocol for giving language models stateful access to tools. It is different from simple REST APIs because it maintains context. Think of it as a two-way conversation between the model and a tool service, not a one-off HTTP request.

Why does this matter? Suppose you want an agent to manage a GitHub repository. With a REST API, it calls endpoints like GET /repos/{owner}/{repo}. With MCP, it opens a persistent connection to a GitHub MCP server and can maintain state--like "I am inside this repository, now list the files in /src."

Open WebUI added native MCP support in v0.6.31. You can connect MCP servers directly without a proxy. This means zero latency compared to older workarounds.

Setting up MCP servers:

Deploy an MCP server (either your own or from the ecosystem). Popular examples include GitHub MCP, Slack MCP, and file system MCP.
In Open WebUI Admin > Integrations > MCP, add the server URL and authentication credentials.
Enable "stateful mode" if the server requires it (most do).
In your agent settings, grant access to the MCP server's tools.
The agent now has persistent access. It can reason about state and execute multi-step workflows.

Authentication: MCP servers support Bearer tokens and OAuth 2.1. Opsily recommends using environment variables for secrets (never hardcode them). Store your GitHub token in GITHUB_TOKEN, pass it to Open WebUI via docker-compose, and reference it in the MCP config.

MCP is optional. You do not need it for simple agents. But for workflows involving stateful services (Git, databases, message queues), it dramatically reduces complexity.

Terminal and Code Execution Setup

Many agents need to run commands or write code. Open WebUI cannot do this natively for security reasons (letting a language model execute arbitrary code on your server is dangerous). Instead, you connect a separate Open Terminal Docker container.

What Open Terminal does: It is a sandboxed environment where agents can execute terminal commands, Python, Node.js, or any language you install. The container is isolated from your main server, so a malicious agent cannot access your database credentials or other secrets.

Setup steps:

Deploy Open Terminal as a separate Docker container. Use an official image or build your own with required libraries (e.g., pandas, requests, git).
Generate a WEBUI_SECRET_KEY (a random string) and share it between Open WebUI and Open Terminal via environment variables.
In Open WebUI Admin > Integrations > Open Terminal, add the container URL and secret.
Create an agent, enable "Execute Code" in the tools panel, and select "Open Terminal" as the execution backend.
The agent can now run code. Try a simple task first--like "write a Python script to list all CSV files"--to verify the connection.

Custom Docker images: The default Open Terminal image includes Python and basic tools. For data science work, build an image with pandas, numpy, matplotlib. For DevOps, add Terraform, kubectl, Helm. This is just a Dockerfile.

Security considerations: Open Terminal runs commands as the container user, not root (if you configure it correctly). Tools like firejail or AppArmor add extra isolation. But the safest approach is to never expose Open Terminal to the public internet--keep it on your private network and access it only through Open WebUI.

Practical Agent Examples

Let us walk through three real scenarios to show how agents solve actual problems.

Example 1: File Processing Agent

You receive CSV files from a partner and need to transform them daily. Manually doing this takes 30 minutes. An agent can do it in seconds.

Setup: Create a custom agent with Llama 3.1 70B (or Claude). System prompt: "You are a data engineer. You can read CSV files, transform data with Python, and write results to S3. Always validate data types before writing."

Tools: File read/write, Python code execution, S3 access.

Workflow: "Process the file 2026-11-sales.csv, remove duplicate rows, convert dates to ISO format, and upload to s3://output/processed/." The agent reads the file, writes Python code to clean it, executes the code, verifies the output, and uploads the result.

Common failure: The agent tries to read the file before checking if it exists. Solution: Add a "file exists" check in the system prompt and provide examples of robust file handling.

Example 2: Research and Summarization Agent

You need market research for a competitor. Instead of clicking through websites, an agent gathers it.

Setup: Custom agent with web search and RAG (knowledge base). Base model: Claude 3.5 Sonnet (best at research tasks). System prompt: "You are a business analyst. Search for information about [company], analyze their pricing, features, and market position. Cross-reference multiple sources and cite them."

Tools: Web search, document parsing, RAG over your internal CRM data.

Workflow: "Research Notion and compare their pricing to Airtable." The agent searches both sites, parses pricing pages, cross-references with your knowledge base (if you have older pricing data), and delivers a structured comparison.

Common failure: The agent returns outdated information or conflates competitor details. Solution: Add recency filters to search queries ("pricing as of 2026") and require multiple source verification in the prompt.

Example 3: Git and Deployment Agent

For DevOps teams, an agent that can open PRs, run tests, and deploy is valuable.

Setup: Custom agent with GitHub MCP, code execution (Open Terminal), and pipeline support. System prompt: "You are a senior engineer. You can create branches, commit code, open PRs, and trigger CI/CD. Always wait for tests to pass before merging."

Tools: Git commands via MCP, terminal for running tests, CI/CD API.

Workflow: "Create a branch for upgrading the database driver, run tests, and open a PR for review." The agent clones the repo, edits the dependency file, runs the test suite, commits, and opens a PR with a summary.

Common failure: The agent opens a PR without running tests first. Solution: Hard constraint in the prompt: "Never open a PR until tests have run and passed. Always check CI status." Also use MCP state to track whether tests have been triggered.

Enterprise Considerations and Troubleshooting

Setting up agents for a team or production environment adds complexity.

Model Capability Requirements: Not every model can run agents. Rule of thumb: if a model cannot pass intermediate reasoning benchmarks, it cannot reason about tool use. Llama 3.1 7B is the minimum for reliable tool calling. Llama 3.1 70B and Claude 3.5 Sonnet are excellent. Avoid older or specialized models (like code-specific or instruction-tuned models) unless you test them first.

Auditability and Approval Gates: In regulated industries, you need logs of what agents did. Track which tools were called, what parameters were passed, and what the outcome was. Open WebUI logs these in the database. Export them regularly to an audit trail.

For sensitive operations (like database updates or Git commits), require human approval before execution. Open WebUI supports workflow approvals--set a flag on sensitive tools to pause execution until a human confirms.

Tool Isolation: Do not let agents access secrets or sensitive data they do not need. If an agent only needs to read data, revoke write access. Use IAM roles and secrets management (HashiCorp Vault, AWS Secrets Manager) to limit exposure. Open Terminal should never have access to your production database password, only a read-only replica password.

Common Failures and Fixes:

"Agent hangs indefinitely": Usually a wrong connection type (agent expects REST, gets WebSocket). Check the integration settings. Also verify your firewall allows outbound connections from the container.
"Agent calls the wrong tool repeatedly": The model is confused. Improve the system prompt with examples of correct tool use. Sometimes a model upgrade (better base model) is needed.
"WEBUI_SECRET_KEY mismatch error": The Open WebUI and Open Terminal containers do not share the same secret. Verify they both read from the same environment variable.
"Tool returns 500 error": The external service is down or rate-limited. Add retry logic in the system prompt: "If a tool fails, wait 5 seconds and retry."
"Agent loses context between messages": Memory is disabled or the model ran out of context window. Enable conversation memory in the agent settings. For long conversations, summarize history and pass summaries instead of full logs.

Scaling from DIY to Managed Hosting: If you are running Open WebUI in a Docker container on your laptop, agents work fine for testing. But for production--multiple users, frequent agent runs, SLAs--you need infrastructure.

Managed Open WebUI hosting handles updates, backups, and scaling automatically. Opsily can host Open WebUI with built-in integrations for Open Terminal, MCP servers, and persistent data storage, so your agents stay reliable and your team avoids infrastructure overhead.

Frequently Asked Questions

How do agents work in Open WebUI?

Agents use a loop: the model generates reasoning, decides which tool to call, executes the tool, reads the result, and repeats until it solves the problem. The system prompt and tool descriptions guide the model's decisions. Memory lets agents track what they have already done to avoid loops.

What models support agent tool calling?

Llama 3.1 7B and larger, Claude 3.5 Sonnet, Mistral Large, and GPT-4 all handle tool calling reliably. Smaller models (under 7B) and older models (GPT-3.5) frequently fail. Test before deploying agents with new models.

Should I use an external agent framework or a custom agent?

Use external frameworks (Hermes, OpenClaw, Computer) if your workflow matches their design. Use custom agents if you need full control over tools and reasoning. Hybrid setups (different agents for different tasks) are common in production.

What is MCP and why would I use it?

MCP (Model Context Protocol) enables stateful tool access--agents maintain context with services like GitHub or Slack across multiple calls. Use it for workflows requiring sequential state changes. Simple REST API calls do not need MCP.

How do I prevent agents from making mistakes or accessing sensitive data?

Use tool-level permissions (read-only access), approval gates for sensitive operations, and isolated containers (Open Terminal) that do not have access to production secrets. Log all agent actions for auditability.

How long does it take to set up an agent from scratch?

A simple custom agent takes 30 minutes (base model, system prompt, one tool). A production agent with multiple tools, memory, and security checks takes a few hours. External agents like Hermes or Computer take 15 minutes because they are pre-built.

Can I run multiple agents in parallel?

Yes. You can create separate agent profiles and queue tasks to different agents. Open WebUI does not enforce sequential execution. For heavy workloads, run agents in separate containers and distribute load with a task queue (e.g., Celery or Apache Airflow).

What should I do if an agent gets stuck in a loop?

Add a step limit in the system prompt ("execute at most 10 steps"). Monitor agent runs and kill long-running ones manually. Improve the base model or system prompt to reduce confusion. Log the conversation to identify where the agent went wrong.

The Bottom Line

Open WebUI agents turn language models into autonomous workers that can handle real workflows. You have three paths: external frameworks (fast, less flexible), custom agents (slower setup, full control), or hybrid (best for teams). Terminal execution and MCP add power but require careful security setup. For production use, hosted infrastructure saves you from managing Docker containers and keeps agents reliable.

Start with a small agent on a single task. Master the system prompt and tool selection. Then scale to production when you are confident. Opsily can host Open WebUI with all the integrations agents need, so your team focuses on building workflows instead of managing servers.