Iyeoluwa Akinrinola
127 lines
3.6 KiB
Markdown
127 lines
3.6 KiB
Markdown
# OpenWebUI Backend Service
|
|
|
|
A backend service for OpenWebUI that provides OpenWebUI-compatible API endpoints for chat functionality and model switching.
|
|
|
|
## Features
|
|
|
|
- OpenWebUI-compatible API endpoints
|
|
- Ollama API proxy
|
|
- Chat functionality with model switching
|
|
- Support for multiple LLM models (gemma3, llama3.3, llama3.1, mistral, deepseek)
|
|
- Team chat functionality
|
|
|
|
## Technology Stack
|
|
|
|
- **Backend**: FastAPI
|
|
- **Chat Models**: Ollama models via OpenWebUI
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
ai_service/
|
|
├── models/ # Model and chat services
|
|
│ ├── model_service.py
|
|
│ ├── chat_service.py
|
|
│ └── model_parameters.py
|
|
├── embeddings/ # Document processing for RAG
|
|
│ └── document_service.py
|
|
├── openwebui_api.py # OpenWebUI-compatible API endpoints
|
|
├── config.py # Configuration settings
|
|
├── api.py # FastAPI application
|
|
└── deploy.sh # Deployment script
|
|
openwebui-bot/ # OpenWebUI bot integration
|
|
run_ai_service.py # Script to run the service
|
|
run_openwebui_bot.py # Script to run the OpenWebUI bot
|
|
run_custom_bot.py # Script to run a custom OpenWebUI bot
|
|
deploy_ai_service.sh # Local deployment script
|
|
remote_deploy.sh # Remote deployment script
|
|
```
|
|
|
|
## Setup Instructions
|
|
|
|
1. Create a virtual environment:
|
|
```
|
|
python -m venv venv
|
|
source venv/bin/activate # On Windows: venv\Scripts\activate
|
|
```
|
|
|
|
2. Install dependencies:
|
|
```
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
3. Copy `.env.example` to `.env` and update the values:
|
|
```
|
|
cp ai_service/.env.example ai_service/.env
|
|
# Edit the .env file with appropriate values
|
|
```
|
|
|
|
4. Run the service:
|
|
```
|
|
python run_ai_service.py
|
|
```
|
|
|
|
## Deployment
|
|
|
|
To deploy the service:
|
|
|
|
1. Local deployment:
|
|
```
|
|
./deploy_ai_service.sh
|
|
```
|
|
|
|
2. Remote deployment:
|
|
```
|
|
./remote_deploy.sh [server_ip] [user] [port] [remote_dir]
|
|
```
|
|
|
|
## API Documentation
|
|
|
|
Once the service is running, you can access the API documentation at:
|
|
- Swagger UI: http://localhost:5252/docs
|
|
- ReDoc: http://localhost:5252/redoc
|
|
|
|
## OpenWebUI Configuration
|
|
|
|
To configure OpenWebUI to use this service as its backend:
|
|
|
|
1. Set the following environment variables in OpenWebUI:
|
|
```
|
|
OLLAMA_API_BASE_URL=http://your-server-ip:5252/ollama
|
|
OPENAI_API_BASE_URL=http://your-server-ip:5252/api
|
|
```
|
|
|
|
2. Restart OpenWebUI to apply the changes.
|
|
|
|
## OpenWebUI Bot Integration
|
|
|
|
This project includes integration with the [Open WebUI bot](https://github.com/open-webui/bot) to provide AI responses in OpenWebUI channels. See [BOT_README.md](BOT_README.md) for detailed instructions on how to use the bot.
|
|
|
|
### Bot Integration
|
|
|
|
The bot is integrated with the AI service and starts automatically when the service starts (if `BOT_ENABLED=true` in the `.env` file). The bot connects to OpenWebUI via WebSocket and listens for messages in channels. When a message mentions the AI (using trigger words like `@ai`), the bot processes the message and sends a response back to the channel.
|
|
|
|
### Bot API Endpoints
|
|
|
|
The AI service provides the following endpoints to manage the bot:
|
|
|
|
- `GET /bot/status`: Get the status of the bot (enabled and running)
|
|
- `POST /bot/start`: Start the bot if it's not already running
|
|
- `POST /bot/stop`: Stop the bot if it's running
|
|
|
|
You can also check the bot status in the configuration endpoint:
|
|
|
|
- `GET /config`: Get the current configuration, including bot status
|
|
|
|
### Running the Bot Separately
|
|
|
|
If you want to run the bot separately from the AI service, you can use the provided scripts:
|
|
|
|
```bash
|
|
# Run the original AI example
|
|
./run_openwebui_bot.py
|
|
|
|
# Or run our custom AI bot implementation
|
|
./run_custom_bot.py
|
|
```
|