ds_mcp_template/tests/README.md

# Test Suite

This directory contains comprehensive tests for the MCP Template system.

## Test Structure

```
tests/
├── unit/                    # Unit tests (fast, no external dependencies)
├── integration/            # Integration tests (require real server connections)
├── e2e/                    # End-to-end tests
└── README.md              # This file
```

## Test Types

### Unit Tests
- **Fast**: Run in milliseconds
- **Isolated**: No external dependencies
- **Focused**: Test individual components
- **Run with**: `python run_tests.py unit`

### Integration Tests
- **Real connections**: Test with actual MCP server
- **API calls**: May require API keys
- **Slower**: Take longer to execute
- **Run with**: `python run_tests.py integration`

## Running Tests

### Quick Start

```bash
# Run all tests
python run_tests.py all

# Run unit tests only
python run_tests.py unit

# Run integration tests only
python run_tests.py integration

# Run specific test file
python run_tests.py specific tests/integration/test_mcp_integration.py
```

### Advanced Options

```bash
# Run with coverage report
python run_tests.py all --coverage

# Skip slow tests
python run_tests.py integration --skip-slow

# Run specific test markers
pytest -m "integration and not slow"
pytest -m "unit"
pytest -m "requires_api_key"
```

## Test Markers

- **`@pytest.mark.unit`**: Unit tests (fast, isolated)
- **`@pytest.mark.integration`**: Integration tests (require server)
- **`@pytest.mark.requires_api_key`**: Tests requiring API keys
- **`@pytest.mark.slow`**: Slow-running tests
- **`@pytest.mark.asyncio`**: Async tests

## Environment Setup

### API Keys (for integration tests)

Create a `.env` file in the project root:

```bash
# OpenAI (required for most tests)
OPENAI_API_KEY=your_openai_api_key_here

# Optional: Other providers
ANTHROPIC_API_KEY=your_anthropic_key_here
GROK_API_KEY=your_grok_key_here
```

### Test Configuration

Tests are configured via `pytest.ini`:
- Async mode enabled
- Custom markers defined
- Test discovery patterns
- Warning filters

## Integration Test Details

### MCP Server Management
Integration tests automatically:
1. Start MCP server processes
2. Wait for server readiness
3. Connect test clients
4. Clean up processes after tests

### Transport Testing
- **SSE Transport**: HTTP-based, tested with real server
- **STDIO Transport**: Direct process communication

### AI Provider Testing
- **OpenAI**: Primary provider for most tests
- **Claude**: Tested if ANTHROPIC_API_KEY available
- **Grok**: Tested if GROK_API_KEY available

## Writing Tests

### Unit Test Example
```python
import pytest
from mcp_template.core.types import TransportType

@pytest.mark.unit
def test_transport_enum():
    """Test TransportType enum values"""
    assert TransportType.SSE.value == "sse"
    assert TransportType.STDIO.value == "stdio"
```

### Integration Test Example
```python
import pytest
from mcp_llm_client import MCPAIClient, TransportType

@pytest.mark.integration
@pytest.mark.requires_api_key
@pytest.mark.asyncio
async def test_client_connection(sse_server):
    """Test MCP client can connect to server"""
    if not os.getenv("OPENAI_API_KEY"):
        pytest.skip("OpenAI API key not available")

    client = MCPAIClient(model="gpt-4o", transport=TransportType.SSE)

    try:
        await client.connect(sse_server.url)
        tools = await client.get_mcp_tools()
        assert len(tools) > 0
    finally:
        await client.disconnect()
```

## Test Coverage

Run tests with coverage:

```bash
# Generate HTML coverage report
python run_tests.py all --coverage

# View coverage report
open htmlcov/index.html
```

## Troubleshooting

### Common Issues

1. **Server won't start**: Check port availability
2. **API key errors**: Ensure `.env` file exists
3. **Connection timeouts**: Increase timeout values
4. **Process cleanup**: Tests handle cleanup automatically

### Debug Mode

```bash
# Run with verbose output
pytest -v -s tests/integration/test_mcp_integration.py::TestMCPClientIntegration::test_sse_transport_connection

# Run single test with debugging
pytest -v --pdb tests/integration/test_mcp_integration.py -k "test_sse_transport"
```

## CI/CD Integration

Tests are designed to work in CI/CD environments:
- Automatic skipping when API keys unavailable
- Process cleanup on test failure
- No manual server startup required
- Cross-platform compatibility

## Performance Testing

Some tests measure performance:
- Connection establishment time
- Query response times
- Concurrent connection handling
- Resource cleanup efficiency

Run performance tests:
```bash
pytest -m "slow" -v
```