ds_zagres_ai/OLLAMA_INTEGRATION.md

Ollama and OpenWebUI Integration

This document explains how to use the chatbot with Ollama and OpenWebUI.

Overview

The chatbot has been updated to use Ollama models via OpenWebUI. The following models are now available:

• gemma3: Google Gemma 3 model
• llama3.3: Meta Llama 3 70B model
• llama3.1: Meta Llama 3 8B model
• mistral: Mistral AI model
• deepseek: DeepSeek model

OpenWebUI

OpenWebUI is running at: http://104.225.217.215:8080/

Features

1. Document Training: OpenWebUI provides built-in RAG capabilities. When you upload a document through OpenWebUI, it automatically processes it for RAG.

2. Knowledge Database: OpenWebUI stores documents in its knowledge database, which can be accessed during chat sessions.

Using the Integration

Configuration

1. Update your .env file with the following settings:

# OpenWebUI configuration
OPENWEBUI_URL=http://104.225.217.215:8080
OPENWEBUI_API_KEY=your-openwebui-api-key

# Ollama configuration
OLLAMA_API_URL=http://localhost:11434
DEFAULT_MODEL=llama3.1

Document Processing

To use RAG with your documents:

1. Go to the OpenWebUI interface at http://104.225.217.215:8080/
2. Navigate to the Knowledge section
3. Upload your documents
4. OpenWebUI will automatically process them for RAG

Chat with RAG

When using the chatbot API:

1. Set use_rag=True in your chat requests to enable RAG
2. The system will use OpenWebUI's knowledge database to enhance responses

API Usage

The API endpoints remain the same, but now they use Ollama models via OpenWebUI:

# Example: Get a response with RAG
response = chat_service.get_chat_response(
    chat_id="your-chat-id",
    message="Tell me about the documents I uploaded",
    user_id="user123",
    use_rag=True,
    model_id="llama3.1"  # Use one of the Ollama models
)

Troubleshooting

If you encounter issues:

1. Make sure OpenWebUI is accessible at http://104.225.217.215:8080/
2. Check that you have the correct API key if authentication is enabled
3. Verify that the documents are properly uploaded to OpenWebUI's knowledge database

Direct Usage

For direct usage without the API, you can:

1. Go to http://104.225.217.215:8080/
2. Select the model you want to use
3. Upload documents in the Knowledge section
4. Chat with the model and it will use the knowledge database automatically