ai_service/
├── models/           # Model and chat services
│   ├── model_service.py
│   ├── chat_service.py
│   └── model_parameters.py
├── embeddings/       # Document processing for RAG
│   └── document_service.py
├── openwebui_api.py  # OpenWebUI-compatible API endpoints
├── config.py         # Configuration settings
├── api.py            # FastAPI application
└── deploy.sh         # Deployment script
openwebui-bot/        # OpenWebUI bot integration
run_ai_service.py     # Script to run the service
run_openwebui_bot.py  # Script to run the OpenWebUI bot
run_custom_bot.py     # Script to run a custom OpenWebUI bot
deploy_ai_service.sh  # Local deployment script
remote_deploy.sh      # Remote deployment script

Setup Instructions

Create a virtual environment:

python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install dependencies:
```
pip install -r requirements.txt
```

Copy .env.example to .env and update the values:

cp ai_service/.env.example ai_service/.env
# Edit the .env file with appropriate values

Run the service:
```
python run_ai_service.py
```

Deployment

To deploy the service:

Local deployment:
```
./deploy_ai_service.sh
```

Remote deployment:

./remote_deploy.sh [server_ip] [user] [port] [remote_dir]

API Documentation

Once the service is running, you can access the API documentation at:

Swagger UI: http://localhost:5252/docs
ReDoc: http://localhost:5252/redoc

OpenWebUI Configuration

To configure OpenWebUI to use this service as its backend:

Set the following environment variables in OpenWebUI:

OLLAMA_API_BASE_URL=http://your-server-ip:5252/ollama
OPENAI_API_BASE_URL=http://your-server-ip:5252/api

Restart OpenWebUI to apply the changes.

OpenWebUI Bot Integration

This project includes integration with the Open WebUI bot to provide AI responses in OpenWebUI channels. See BOT_README.md for detailed instructions on how to use the bot.

Bot Integration

The bot is integrated with the AI service and starts automatically when the service starts (if BOT_ENABLED=true in the .env file). The bot connects to OpenWebUI via WebSocket and listens for messages in channels. When a message mentions the AI (using trigger words like @ai), the bot processes the message and sends a response back to the channel.

Bot API Endpoints

The AI service provides the following endpoints to manage the bot:

GET /bot/status: Get the status of the bot (enabled and running)
POST /bot/start: Start the bot if it's not already running
POST /bot/stop: Stop the bot if it's running

You can also check the bot status in the configuration endpoint:

GET /config: Get the current configuration, including bot status

Running the Bot Separately

If you want to run the bot separately from the AI service, you can use the provided scripts:

# Run the original AI example
./run_openwebui_bot.py

# Or run our custom AI bot implementation
./run_custom_bot.py