Tool-Enabled AI Chat (Open WebUI + MCP)

Supercharge your local AI with tools. Open WebUI is the most popular self-hosted ChatGPT interface, and it now supports the Model Context Protocol (MCP) - an open standard that lets your AI connect to filesystems, web searches, databases, APIs, and more. Pair it with Ollama for local inference and you get a ChatGPT-like experience with plugin-style tool calling - all running on your own hardware.

What you get

• ChatGPT-like UI with multi-chat tabs, history, and model switching
• MCP tool calling - your AI can read files, search the web, query databases, and call APIs
• Web search - connect a web search MCP server for up-to-date answers
• File access - read, write, and analyze local files
• Code execution - built-in Python sandbox for data analysis
• RAG - upload documents and ask questions against your knowledge base
• Multi-user - share with your team with role-based access
• 100% private - everything runs on your hardware, no cloud dependency

Architecture

For better tool-calling reliability, a higher quantization level (Q6_K or Q8) is recommended. Recommended GPU: RTX 3060 12GB or RTX 4070 Super for more headroom.

Prerequisites

• GPU with ≥12 GB VRAM (tool calling benefits from higher quant models)
• Docker + Docker Compose
• Node.js + npm (for some MCP servers)
• ~15 GB free disk

Setup

Step 1: Start Ollama

docker run -d --gpus all -p 11434:11434 --name ollama \
  -v ollama:/root/.ollama \
  ollama/ollama

Pull a model with strong function-calling:

docker exec ollama ollama pull qwen3:14b

For better tool-calling accuracy, pull a higher quant:

docker exec ollama ollama pull qwen3:14b:q8_0

Step 2: Start Open WebUI with MCP Enabled

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    depends_on:
      - ollama
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - MCP_ENABLE=true
      - ENABLE_TOOLS=true
    volumes:
      - open-webui:/app/backend/data
    ports:
      - "3000:8080"
    restart: unless-stopped
 
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - ollama:/root/.ollama
    ports:
      - "11434:11434"
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    restart: unless-stopped
 
volumes:
  ollama:
  open-webui:

docker compose up -d

Open http://localhost:3000 and create your admin account.

Step 3: Configure MCP Tools

Go to Admin Panel → Settings → External Tools in Open WebUI.

Here you can add MCP servers. The simplest method is to use the mcpo proxy which wraps MCP servers as OpenAI-compatible endpoints.

Add Web Search (Brave Search MCP)

# Install the Brave Search MCP server
npx -y @anthropic/mcp-server-brave-search

In Open WebUI External Tools, add a new tool pointing to the MCP server.

Add Filesystem Access

Filesystem MCP lets your AI read and write files in allowed directories:

npx -y @modelcontextprotocol/server-filesystem /allowed/path

Use Pre-Built Tool Collections

Open WebUI has a community tools library. Browse available tools from within the admin panel.

Step 4: Use Tools in Chat

1. Start a new chat in Open WebUI
2. Click the tools icon (wrench) next to the input box
3. Select which tools the AI can use this session
4. Ask something that requires a tool: "Search the web for latest AI news" or "Read the data.csv file and summarize it"

The AI will decide when to call a tool, execute it, and incorporate the results into its response.

Use it

Web-Aware Assistant

Enable web search. Ask "What's the latest on the Gemma 4 release?" - the AI searches the web, reads results, and gives you a summary with sources.

Data Analyst

Upload a CSV, enable the code interpreter. Ask "Create a chart of monthly sales trends" - the AI reads the file, writes Python, executes it, and displays the chart.

Documentation Helper

Connect the filesystem MCP to your project's docs folder. Ask "Find the API endpoint for user authentication" - the AI reads your docs and answers from them.

Research Assistant

Enable web search and filesystem access. Ask "Research vector databases and save a summary to research.md" - the AI searches, reads, and writes the file.

Cost vs cloud

Troubleshooting

• Tool calls not working → Make sure MCP_ENABLE=true is set in the Open WebUI environment. Some models need a higher quant for reliable function calling (use q8_0).
• MCP server shows "connection refused" → If running MCP servers on the host, use host.docker.internal instead of localhost from within Docker.
• Model doesn't call tools → Not all models support function calling well. Qwen3 and Llama 3.1 have strong tool-calling. Avoid very small models (<7B).
• Web search returns nothing → Check that the MCP server is running (npx command still active). Add --add-host host.docker.internal:host-gateway to the Open WebUI container.
• Slow with tools → Tool calling adds latency. Use Q4 quant for speed, Q8 for reliability.

Swap components

• MCPO proxy → Use mcpo to wrap MCP servers as OpenAI-compatible endpoints for simpler integration.
• vLLM instead of Ollama → For high throughput, serve models with vLLM which has native MCP tool-calling support.
• Any OpenAI-compatible provider → Open WebUI works with OpenAI, Anthropic, and any OpenAI-compatible endpoint alongside Ollama.
• Desktop app → Use the Open WebUI desktop app instead of Docker for simpler local setup.