88 lines
2.3 KiB
Markdown
88 lines
2.3 KiB
Markdown
|
# Ollama and OpenWebUI Integration
|
||
|
|
|
||
|
|
This document explains how to use the chatbot with Ollama and OpenWebUI.
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
The chatbot has been updated to use Ollama models via OpenWebUI. The following models are now available:
|
||
|
|
|
||
|
|
- **gemma3**: Google Gemma 3 model
|
||
|
|
- **llama3.3**: Meta Llama 3 70B model
|
||
|
|
- **llama3.1**: Meta Llama 3 8B model
|
||
|
|
- **mistral**: Mistral AI model
|
||
|
|
- **deepseek**: DeepSeek model
|
||
|
|
|
||
|
|
## OpenWebUI
|
||
|
|
|
||
|
|
OpenWebUI is running at: http://104.225.217.215:8080/
|
||
|
|
|
||
|
|
### Features
|
||
|
|
|
||
|
|
1. **Document Training**: OpenWebUI provides built-in RAG capabilities. When you upload a document through OpenWebUI, it automatically processes it for RAG.
|
||
|
|
|
||
|
|
2. **Knowledge Database**: OpenWebUI stores documents in its knowledge database, which can be accessed during chat sessions.
|
||
|
|
|
||
|
|
## Using the Integration
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
|
||
|
|
1. Update your `.env` file with the following settings:
|
||
|
|
|
||
|
|
```
|
||
|
|
# OpenWebUI configuration
|
||
|
|
OPENWEBUI_URL=http://104.225.217.215:8080
|
||
|
|
OPENWEBUI_API_KEY=your-openwebui-api-key
|
||
|
|
|
||
|
|
# Ollama configuration
|
||
|
|
OLLAMA_API_URL=http://localhost:11434
|
||
|
|
DEFAULT_MODEL=llama3.1
|
||
|
|
```
|
||
|
|
|
||
|
|
### Document Processing
|
||
|
|
|
||
|
|
To use RAG with your documents:
|
||
|
|
|
||
|
|
1. Go to the OpenWebUI interface at http://104.225.217.215:8080/
|
||
|
|
2. Navigate to the Knowledge section
|
||
|
|
3. Upload your documents
|
||
|
|
4. OpenWebUI will automatically process them for RAG
|
||
|
|
|
||
|
|
### Chat with RAG
|
||
|
|
|
||
|
|
When using the chatbot API:
|
||
|
|
|
||
|
|
1. Set `use_rag=True` in your chat requests to enable RAG
|
||
|
|
2. The system will use OpenWebUI's knowledge database to enhance responses
|
||
|
|
|
||
|
|
## API Usage
|
||
|
|
|
||
|
|
The API endpoints remain the same, but now they use Ollama models via OpenWebUI:
|
||
|
|
|
||
|
|
```python
|
||
|
|
# Example: Get a response with RAG
|
||
|
|
response = chat_service.get_chat_response(
|
||
|
|
chat_id="your-chat-id",
|
||
|
|
message="Tell me about the documents I uploaded",
|
||
|
|
user_id="user123",
|
||
|
|
use_rag=True,
|
||
|
|
model_id="llama3.1" # Use one of the Ollama models
|
||
|
|
)
|
||
|
|
```
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
If you encounter issues:
|
||
|
|
|
||
|
|
1. Make sure OpenWebUI is accessible at http://104.225.217.215:8080/
|
||
|
|
2. Check that you have the correct API key if authentication is enabled
|
||
|
|
3. Verify that the documents are properly uploaded to OpenWebUI's knowledge database
|
||
|
|
|
||
|
|
## Direct Usage
|
||
|
|
|
||
|
|
For direct usage without the API, you can:
|
||
|
|
|
||
|
|
1. Go to http://104.225.217.215:8080/
|
||
|
|
2. Select the model you want to use
|
||
|
|
3. Upload documents in the Knowledge section
|
||
|
|
4. Chat with the model and it will use the knowledge database automatically
|