Iyeoluwa Akinrinola
161 lines
3.7 KiB
Markdown
161 lines
3.7 KiB
Markdown
# Local Testing with Remote Server Resources
|
|
|
|
This guide explains how to set up local testing that connects to remote server resources for the chatbot project.
|
|
|
|
## Overview
|
|
|
|
These scripts allow you to:
|
|
|
|
1. Test the connection to the remote Ollama server
|
|
2. Create SSH tunnels to access remote resources locally
|
|
3. Run tests against the remote Ollama models from your local machine
|
|
|
|
## Setup Instructions
|
|
|
|
### 1. Initial Setup
|
|
|
|
Run the setup script to create a virtual environment and install dependencies:
|
|
|
|
```bash
|
|
chmod +x setup_local_test_env.sh
|
|
./setup_local_test_env.sh
|
|
```
|
|
|
|
This will:
|
|
- Create a Python virtual environment
|
|
- Install required dependencies
|
|
- Create a `local_test.env` file from the template
|
|
|
|
### 2. Configure Environment Variables
|
|
|
|
Edit the `local_test.env` file with your server credentials:
|
|
|
|
```bash
|
|
# Remote server configuration
|
|
SERVER_IP=104.225.217.215
|
|
SERVER_PORT=22
|
|
SERVER_USER=root
|
|
SERVER_PASSWORD=your_password_here # Add your actual password
|
|
|
|
# Ollama configuration
|
|
OLLAMA_API_URL=http://104.225.217.215:11434
|
|
|
|
# OpenWebUI configuration
|
|
OPENWEBUI_URL=http://104.225.217.215:8080
|
|
OPENWEBUI_API_KEY=your_openwebui_api_key_here
|
|
```
|
|
|
|
### 3. Create SSH Tunnels (Optional)
|
|
|
|
If you want to access the remote resources through localhost (recommended for security and to avoid firewall issues), create SSH tunnels:
|
|
|
|
```bash
|
|
chmod +x create_ssh_tunnel.sh
|
|
./create_ssh_tunnel.sh
|
|
```
|
|
|
|
This will create SSH tunnels for:
|
|
- Ollama API (localhost:11434 → remote:11434)
|
|
- OpenWebUI (localhost:8080 → remote:8080)
|
|
|
|
If you use SSH tunnels, update your `local_test.env` file to use localhost URLs:
|
|
|
|
```bash
|
|
# Ollama configuration
|
|
OLLAMA_API_URL=http://localhost:11434
|
|
|
|
# OpenWebUI configuration
|
|
OPENWEBUI_URL=http://localhost:8080
|
|
```
|
|
|
|
### 4. Run Tests
|
|
|
|
Activate the virtual environment and run the test script:
|
|
|
|
```bash
|
|
source venv/bin/activate
|
|
python test_remote_ollama.py
|
|
```
|
|
|
|
#### Test Options
|
|
|
|
You can customize the test with command-line arguments:
|
|
|
|
```bash
|
|
# Test with a specific model
|
|
python test_remote_ollama.py --model llama3.3
|
|
|
|
# Test with a custom prompt
|
|
python test_remote_ollama.py --prompt "Explain quantum computing"
|
|
|
|
# Test with a different Ollama URL
|
|
python test_remote_ollama.py --ollama-url http://your-server-ip:11434
|
|
|
|
# Test with a different timeout
|
|
python test_remote_ollama.py --timeout 600
|
|
```
|
|
|
|
### 5. Stop SSH Tunnels
|
|
|
|
When you're done testing, stop the SSH tunnels:
|
|
|
|
```bash
|
|
chmod +x stop_ssh_tunnels.sh
|
|
./stop_ssh_tunnels.sh
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Connection Issues
|
|
|
|
If you can't connect to the remote server:
|
|
|
|
1. Check if the server is reachable:
|
|
```bash
|
|
ping 104.225.217.215
|
|
```
|
|
|
|
2. Verify that Ollama is running on the server:
|
|
```bash
|
|
ssh root@104.225.217.215 "curl http://localhost:11434/api/tags"
|
|
```
|
|
|
|
3. Check if there are firewall rules blocking the connection:
|
|
```bash
|
|
ssh root@104.225.217.215 "iptables -L"
|
|
```
|
|
|
|
### SSH Tunnel Issues
|
|
|
|
If the SSH tunnels aren't working:
|
|
|
|
1. Make sure you have SSH access to the server:
|
|
```bash
|
|
ssh root@104.225.217.215
|
|
```
|
|
|
|
2. Check if the ports are already in use:
|
|
```bash
|
|
netstat -tuln | grep 11434
|
|
netstat -tuln | grep 8080
|
|
```
|
|
|
|
3. Try creating the tunnels manually:
|
|
```bash
|
|
ssh -N -L 11434:localhost:11434 root@104.225.217.215
|
|
```
|
|
|
|
## Advanced Usage
|
|
|
|
### Testing with OpenWebUI's RAG Capabilities
|
|
|
|
To test document-based question answering using OpenWebUI's knowledge database:
|
|
|
|
```bash
|
|
python test_remote_ollama.py --prompt "What information do you have about our project?" --model llama3.1
|
|
```
|
|
|
|
### Integrating with Your Local Development
|
|
|
|
You can use these scripts as a foundation for local development that connects to the remote resources. This allows you to develop and test locally while using the remote server's models and data.
|