Update matching logic: AI scores all candidates, lower threshold, absolute amount, prompt improvements
This commit is contained in:
Iyeoluwa Akinrinola
@@ -0,0 +1,220 @@
|
||||
# AI Bookkeeper - Data Science Engine
|
||||
|
||||
AI-powered receipt-to-transaction matching engine using Groq LLM. This is a **Data Science Engine** that provides intelligent matching capabilities for backend applications.
|
||||
|
||||
## 🎯 Purpose
|
||||
|
||||
This Data Science Engine receives QuickBooks transaction data from backend applications and provides:
|
||||
- **AI-powered receipt processing** (OCR and data extraction)
|
||||
- **Intelligent receipt-transaction matching** with confidence scores
|
||||
- **Google Drive integration** for batch receipt processing
|
||||
- **Configurable AI rules** for business logic
|
||||
- **Feedback logging** for continuous improvement
|
||||
|
||||
## 🚀 Quick Start
|
||||
|
||||
### 1. Install Dependencies
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. Configure API Keys
|
||||
The Groq API key is already configured in `config.py`
|
||||
|
||||
### 3. Start the DS Engine
|
||||
```bash
|
||||
python main.py
|
||||
```
|
||||
|
||||
### 4. Access API Documentation
|
||||
- **Swagger UI**: http://localhost:8343/docs
|
||||
- **ReDoc**: http://localhost:8343/redoc
|
||||
|
||||
## 📋 API Endpoints
|
||||
|
||||
### QuickBooks Data Import
|
||||
- `POST /transactions/import/quickbooks` - Import and convert QuickBooks transactions
|
||||
|
||||
### Receipt Processing
|
||||
- `POST /upload` - Upload receipt documents (PDF/images)
|
||||
- `POST /process/{file_id}` - Extract data from uploaded documents
|
||||
- `GET /documents` - List all processed documents
|
||||
|
||||
### Google Drive Integration
|
||||
- `POST /drive/sync` - Sync and process receipts from Google Drive
|
||||
- `GET /drive/folders` - List accessible Google Drive folders
|
||||
- `GET /drive/folder/{folder_id}` - Get folder information
|
||||
|
||||
### AI Matching Engine
|
||||
- `POST /match` - Match receipts to transactions using AI
|
||||
- `POST /approve` - Approve or reject AI matches
|
||||
|
||||
### AI Rules Management
|
||||
- `POST /rules` - Add new AI rules
|
||||
- `GET /rules` - List all active rules
|
||||
- `DELETE /rules/{rule_name}` - Delete rules
|
||||
|
||||
### System Monitoring
|
||||
- `GET /stats` - Get system statistics and performance metrics
|
||||
|
||||
## 🔧 Core Components
|
||||
|
||||
### **AIMatcher** (`ai_matcher.py`)
|
||||
- Uses Groq LLM to compare receipts and transactions
|
||||
- Provides confidence scores and reasoning
|
||||
- Configurable matching criteria (amount, date, vendor)
|
||||
|
||||
### **AIRulesEngine** (`ai_rules.py`)
|
||||
- Applies business rules for auto-approval and categorization
|
||||
- Configurable rule conditions and actions
|
||||
- Supports system and user-generated rules
|
||||
|
||||
### **DocumentProcessor** (`document_processor.py`)
|
||||
- AI-powered receipt data extraction
|
||||
- Supports PDF and image formats
|
||||
- Uses Groq vision model for OCR
|
||||
|
||||
### **MatchingEngine** (`matching_engine.py`)
|
||||
- Main orchestrator combining all components
|
||||
- Handles the complete matching workflow
|
||||
- Provides statistics and feedback logging
|
||||
|
||||
### **FeedbackLogger** (`feedback_logger.py`)
|
||||
- Tracks manual overrides for AI training
|
||||
- Maintains audit trail of user decisions
|
||||
- Enables continuous model improvement
|
||||
|
||||
## 📊 Configuration
|
||||
|
||||
Edit `config.py` to adjust:
|
||||
- **Confidence threshold** (default: 0.8)
|
||||
- **Date tolerance days** (default: 7)
|
||||
- **Amount tolerance percent** (default: 5%)
|
||||
- **Groq API key** (already configured)
|
||||
|
||||
## 🔄 Integration Workflow
|
||||
|
||||
### 1. Backend Sends QuickBooks Data
|
||||
```python
|
||||
# Backend sends QuickBooks transactions
|
||||
response = requests.post(
|
||||
"http://localhost:8343/transactions/import/quickbooks",
|
||||
json={
|
||||
"transactions": [
|
||||
{
|
||||
"id": "QB_TXN_123",
|
||||
"txn_date": "2024-01-15",
|
||||
"amount": 12.50,
|
||||
"payee_name": "Starbucks",
|
||||
"memo": "Coffee purchase"
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### 2. Process Receipts
|
||||
```python
|
||||
# Sync from Google Drive
|
||||
response = requests.post(
|
||||
"http://localhost:8343/drive/sync",
|
||||
json={"folder_id": "your_folder_id"}
|
||||
)
|
||||
|
||||
# Or upload directly
|
||||
response = requests.post(
|
||||
"http://localhost:8343/upload",
|
||||
files={"file": receipt_file}
|
||||
)
|
||||
```
|
||||
|
||||
### 3. AI Matching
|
||||
```python
|
||||
# Match receipts to transactions
|
||||
response = requests.post(
|
||||
"http://localhost:8343/match",
|
||||
json={
|
||||
"receipts": processed_receipts,
|
||||
"transactions": converted_transactions
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
### 4. User Feedback
|
||||
```python
|
||||
# Approve or reject matches
|
||||
response = requests.post(
|
||||
"http://localhost:8343/approve",
|
||||
json={
|
||||
"match_id": "match_123",
|
||||
"user_id": "user_456",
|
||||
"action": "approve"
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
## 🎯 Key Features
|
||||
|
||||
- **AI-powered matching** with confidence scores
|
||||
- **Rule-based auto-approval** and categorization
|
||||
- **Feedback logging** for continuous improvement
|
||||
- **Configurable matching parameters**
|
||||
- **Google Drive integration** for batch processing
|
||||
- **JSON API** for easy backend integration
|
||||
- **Comprehensive error handling**
|
||||
|
||||
## 📝 Data Formats
|
||||
|
||||
### QuickBooks Transaction Input
|
||||
```json
|
||||
{
|
||||
"id": "string",
|
||||
"txn_date": "YYYY-MM-DD",
|
||||
"amount": 0.00,
|
||||
"payee_name": "string",
|
||||
"memo": "string (optional)",
|
||||
"account_name": "string (optional)",
|
||||
"txn_type": "string (optional)"
|
||||
}
|
||||
```
|
||||
|
||||
### Match Result Output
|
||||
```json
|
||||
{
|
||||
"receipt_id": "string",
|
||||
"transaction_id": "string",
|
||||
"confidence_score": 0.95,
|
||||
"match_reason": "string",
|
||||
"receipt_vendor": "string",
|
||||
"receipt_amount": 0.00,
|
||||
"transaction_vendor": "string",
|
||||
"transaction_amount": 0.00
|
||||
}
|
||||
```
|
||||
|
||||
## 🔍 AI Matching Criteria
|
||||
|
||||
The engine uses three primary criteria for matching:
|
||||
|
||||
1. **Amount Similarity** - Compares receipt and transaction amounts (5% tolerance)
|
||||
2. **Date Proximity** - Checks date closeness (7-day tolerance)
|
||||
3. **Vendor Matching** - AI-powered vendor name comparison
|
||||
|
||||
## 🚀 Production Deployment
|
||||
|
||||
For production deployment:
|
||||
- Replace in-memory storage with a database
|
||||
- Configure proper authentication
|
||||
- Set up monitoring and logging
|
||||
- Use environment variables for configuration
|
||||
- Implement proper error handling and retries
|
||||
|
||||
## 📞 Support
|
||||
|
||||
This Data Science Engine is designed to be integrated with backend applications that handle:
|
||||
- QuickBooks API connections
|
||||
- User interface and workflows
|
||||
- Data persistence and management
|
||||
- External integrations
|
||||
|
||||
The engine focuses purely on AI/ML capabilities and provides a clean JSON API for backend integration.
|
||||
Reference in New Issue
Block a user