boladeE
696 lines
34 KiB
Plaintext
696 lines
34 KiB
Plaintext
2025-04-21 20:33:27,700 - root - INFO - Processing upload for document ID: 78afc395-9b7c-4388-8d8b-aa1d02fbf75f
|
|
2025-04-21 20:33:27,705 - root - INFO - File saved to data/uploads/78afc395-9b7c-4388-8d8b-aa1d02fbf75f_2.Tender Specifications.docx
|
|
2025-04-21 20:33:27,722 - root - INFO - Processing document 78afc395-9b7c-4388-8d8b-aa1d02fbf75f with content length: 17509
|
|
2025-04-21 20:33:28,023 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 20:33:28,939 - root - INFO - Stored embedding for document 78afc395-9b7c-4388-8d8b-aa1d02fbf75f
|
|
2025-04-21 20:34:54,473 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 20:36:26,368 - root - INFO - Document 78afc395-9b7c-4388-8d8b-aa1d02fbf75f processed successfully
|
|
2025-04-21 20:36:26,384 - root - INFO - Document 78afc395-9b7c-4388-8d8b-aa1d02fbf75f processed successfully
|
|
2025-04-21 21:10:13,818 - root - INFO - Processing upload for document ID: 1130b5e6-c350-40f9-94c2-9ad43d6fa4a4
|
|
2025-04-21 21:10:13,818 - root - INFO - File saved to data/uploads/1130b5e6-c350-40f9-94c2-9ad43d6fa4a4_1.Invitation to Tender.docx
|
|
2025-04-21 21:10:13,834 - root - INFO - Processing document 1130b5e6-c350-40f9-94c2-9ad43d6fa4a4 with content length: 13880
|
|
2025-04-21 21:10:14,176 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 21:10:14,994 - root - INFO - Stored embedding for document 1130b5e6-c350-40f9-94c2-9ad43d6fa4a4
|
|
2025-04-21 21:11:14,319 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 21:11:14,326 - root - ERROR - Error processing document 1130b5e6-c350-40f9-94c2-9ad43d6fa4a4: 'NoneType' object has no attribute 'lower'
|
|
2025-04-21 21:11:14,326 - root - ERROR - Error processing document: 'NoneType' object has no attribute 'lower'
|
|
2025-04-21 21:11:14,326 - root - ERROR - Traceback (most recent call last):
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\main.py", line 112, in upload_document
|
|
await document_processor.process_document(doc_id, file_path, document_type)
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\services\document_processor.py", line 147, in process_document
|
|
"issues": self._format_issues(reranked_issues),
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\services\document_processor.py", line 227, in _format_issues
|
|
if any(word in issue_text.lower() for word in ['critical', 'severe', 'major', 'high risk']):
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\services\document_processor.py", line 227, in <genexpr>
|
|
if any(word in issue_text.lower() for word in ['critical', 'severe', 'major', 'high risk']):
|
|
^^^^^^^^^^^^^^^^
|
|
AttributeError: 'NoneType' object has no attribute 'lower'
|
|
|
|
2025-04-21 21:22:01,931 - root - INFO - Processing upload for document ID: aecbb62c-b7ed-4c2e-beff-fe5e292de9f1
|
|
2025-04-21 21:22:01,940 - root - INFO - File saved to data/uploads/aecbb62c-b7ed-4c2e-beff-fe5e292de9f1_4.Scope of Work.docx
|
|
2025-04-21 21:22:01,994 - root - INFO - Processing document aecbb62c-b7ed-4c2e-beff-fe5e292de9f1 with content length: 15493
|
|
2025-04-21 21:22:02,392 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 21:22:03,409 - root - INFO - Stored embedding for document aecbb62c-b7ed-4c2e-beff-fe5e292de9f1
|
|
2025-04-21 21:22:56,055 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 21:24:51,709 - root - INFO - Document aecbb62c-b7ed-4c2e-beff-fe5e292de9f1 processed successfully
|
|
2025-04-21 21:24:51,717 - root - INFO - Document aecbb62c-b7ed-4c2e-beff-fe5e292de9f1 processed successfully
|
|
2025-04-21 21:34:15,975 - root - INFO - Processing upload for document ID: a444141c-e0c0-4ee9-a448-4555494aaede
|
|
2025-04-21 21:34:16,015 - root - INFO - File saved to data/uploads/a444141c-e0c0-4ee9-a448-4555494aaede_3.Bill of Quantities.docx
|
|
2025-04-21 21:34:16,058 - root - INFO - Processing document a444141c-e0c0-4ee9-a448-4555494aaede with content length: 13346
|
|
2025-04-21 21:34:16,557 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 21:34:18,925 - root - INFO - Stored embedding for document a444141c-e0c0-4ee9-a448-4555494aaede
|
|
2025-04-21 21:35:43,925 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 21:36:09,310 - root - INFO - Document a444141c-e0c0-4ee9-a448-4555494aaede processed successfully
|
|
2025-04-21 21:36:09,328 - root - INFO - Document a444141c-e0c0-4ee9-a448-4555494aaede processed successfully
|
|
2025-04-21 21:38:22,988 - root - ERROR - Error retrieving analysis: list object has no element 1
|
|
2025-04-21 21:38:23,032 - root - ERROR - Traceback (most recent call last):
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\main.py", line 139, in get_analysis
|
|
return templates.TemplateResponse(
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\starlette\templating.py", line 209, in TemplateResponse
|
|
return _TemplateResponse(
|
|
^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\starlette\templating.py", line 40, in __init__
|
|
content = template.render(context)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\jinja2\environment.py", line 1295, in render
|
|
self.environment.handle_exception()
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\jinja2\environment.py", line 942, in handle_exception
|
|
raise rewrite_traceback_stack(source=source)
|
|
File "src\templates\analysis.html", line 1, in top-level template code
|
|
{% extends "base.html" %}
|
|
File "src\templates\base.html", line 34, in top-level template code
|
|
{% block content %}{% endblock %}
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "src\templates\analysis.html", line 50, in block 'content'
|
|
{{ analysis.recommendations[loop.index0]|markdown|safe }}
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\main.py", line 44, in markdown_filter
|
|
return markdown.markdown(text, extensions=['extra', 'nl2br'])
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\markdown\core.py", line 482, in markdown
|
|
return md.convert(text)
|
|
^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\markdown\core.py", line 341, in convert
|
|
if not source.strip():
|
|
^^^^^^^^^^^^
|
|
jinja2.exceptions.UndefinedError: list object has no element 1
|
|
|
|
2025-04-21 21:40:33,659 - root - INFO - Processing upload for document ID: befee86e-cc92-43cf-b935-3c8ec78ef275
|
|
2025-04-21 21:40:33,663 - root - INFO - File saved to data/uploads/befee86e-cc92-43cf-b935-3c8ec78ef275_4.Scope of Work.docx
|
|
2025-04-21 21:40:33,694 - root - INFO - Processing document befee86e-cc92-43cf-b935-3c8ec78ef275 with content length: 15493
|
|
2025-04-21 21:40:34,174 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 21:40:35,491 - root - INFO - Stored embedding for document befee86e-cc92-43cf-b935-3c8ec78ef275
|
|
2025-04-21 21:41:47,586 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 21:42:22,869 - root - INFO - Document befee86e-cc92-43cf-b935-3c8ec78ef275 processed successfully
|
|
2025-04-21 21:42:22,887 - root - INFO - Document befee86e-cc92-43cf-b935-3c8ec78ef275 processed successfully
|
|
2025-04-21 21:53:13,705 - root - INFO - Processing upload for document ID: 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3
|
|
2025-04-21 21:53:13,713 - root - INFO - File saved to data/uploads/3f10b972-3eb6-43c9-bfeb-777d0a33c4a3_8.form of tender.docx
|
|
2025-04-21 21:53:13,755 - root - INFO - Processing document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3 with content length: 16555
|
|
2025-04-21 21:53:16,168 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 21:53:18,290 - root - INFO - Stored embedding for document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3
|
|
2025-04-21 21:54:26,818 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 21:55:06,268 - root - INFO - Document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3 processed successfully
|
|
2025-04-21 21:55:06,268 - root - INFO - Document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3 processed successfully
|
|
2025-04-21 22:11:40,110 - root - INFO - Processing upload for document ID: 2de0becf-fcfc-4793-aad0-7369180c6980
|
|
2025-04-21 22:11:40,113 - root - INFO - File saved to data/uploads/2de0becf-fcfc-4793-aad0-7369180c6980_3.Bill of Quantities.docx
|
|
2025-04-21 22:11:40,158 - root - INFO - Processing document 2de0becf-fcfc-4793-aad0-7369180c6980 with content length: 13346
|
|
2025-04-21 22:11:40,648 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 22:11:41,772 - root - INFO - Stored embedding for document 2de0becf-fcfc-4793-aad0-7369180c6980
|
|
2025-04-21 22:12:35,639 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 22:13:08,662 - root - INFO - Document 2de0becf-fcfc-4793-aad0-7369180c6980 processed successfully
|
|
2025-04-21 22:13:08,672 - root - INFO - Document 2de0becf-fcfc-4793-aad0-7369180c6980 processed successfully
|
|
2025-04-21 22:15:10,060 - root - INFO - Processing upload for document ID: d6c97c26-b59b-4854-bf6d-6f7f58f3dc11
|
|
2025-04-21 22:15:10,060 - root - INFO - File saved to data/uploads/d6c97c26-b59b-4854-bf6d-6f7f58f3dc11_4.Scope of Work.docx
|
|
2025-04-21 22:15:10,140 - root - INFO - Processing document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11 with content length: 15493
|
|
2025-04-21 22:15:10,472 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 22:15:11,322 - root - INFO - Stored embedding for document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11
|
|
2025-04-21 22:16:10,272 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 22:16:43,822 - root - INFO - Analysis for document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11: {
|
|
"document_id": "d6c97c26-b59b-4854-bf6d-6f7f58f3dc11",
|
|
"summary": "Here are the key sections and requirements extracted from the document:\n\n### 1. **Content Types**\n - Defines the types of content within the document (e.g., XML files, relationships, themes).\n\n### 2. **Relationships**\n - Specifies the relationships between different parts of the document, such as how document parts are connected.\n\n### 3. **Document Content**\n - Contains the main content of the document, including text and formatting.\n\n### 4. **Document Relationships**\n - Defines the relationships specific to the main document part.\n\n### 5. **Theme**\n - Contains theme-related settings for the document, including color schemes and fonts.\n\n### 6. **Settings**\n - Includes document settings such as proofing options, compatibility settings, and other Word-specific configurations.\n\n### 7. **Numbering**\n - Defines numbering formats and styles used in the document.\n\n### 8. **Styles**\n - Specifies the styles applied throughout the document, including paragraph and character styles.\n\n### 9. **Web Settings**\n - Contains settings related to how the document is displayed in a web browser.\n\n### 10. **Font Table**\n - Lists the fonts used in the document and their properties.\n\n### 11. **Core Properties**\n - Includes metadata about the document, such as title, author, and creation date.\n\n### 12. **Application Properties**\n - Contains properties specific to the application (e.g., Microsoft Word) used to create the document.\n\n### Key Requirements:\n- The document must adhere to the defined content types and relationships.\n- Proper formatting and styling must be applied as specified in the styles and numbering sections.\n- Theme settings must be consistent throughout the document.\n- Metadata (core and application properties) must be accurately filled out.\n- Web settings should ensure proper display if the document is viewed online.\n\nThis summary provides an overview of the document's structure and key components, which are essential for maintaining consistency and functionality in the Word document.",
|
|
"issues": [
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 1
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 2
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 3
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 4
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 5
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 6
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 7
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 8
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 9
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 10
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 11
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 12
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 13
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 14
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 15
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 16
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 17
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 18
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 19
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 20
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 21
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 22
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 23
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 24
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 25
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 26
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 27
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 28
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 29
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 30
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 31
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 32
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 33
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 34
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 35
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 36
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 37
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 38
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 39
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 40
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 41
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 42
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 43
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 44
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 45
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 46
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 47
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 48
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 49
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 50
|
|
}
|
|
],
|
|
"recommendations": [
|
|
"### **Comprehensive Compliance Recommendation for Document Issues** \n\n#### **1. Missing or Unidentified Documents (Indexes 0, 49, 1, 30, 25)** \n**Issue:** Several documents are either missing (`document=None`) or improperly indexed, leading to potential compliance risks. \n\n**Action Steps:** \n- **Conduct a Document Audit:** \n - Identify all missing documents (`index=0, 49, 1, 30, 25`) and verify if they were misplaced, incorrectly labeled, or never properly stored. \n - Cross-reference with a master document registry to confirm expected files. \n- **Implement a Document Tracking System:** \n - Use a **Document Management System (DMS)** with version control, metadata tagging, and audit trails. \n - Assign unique identifiers (e.g., `DOC-001`, `POL-2024-01`) to prevent indexing errors. \n- **Enforce Mandatory Metadata:** \n - Require fields such as: \n - Document title \n - Version number \n - Creation/modification dates \n - Author/owner \n - Compliance status (e.g., \"Reviewed,\" \"Pending Approval\") \n\n#### **2. Inconsistent Relevance Scores (Ranging from 0.23 to 0.98)** \n**Issue:** The relevance scores vary significantly, indicating potential inconsistencies in document classification, retrieval, or applicability. \n\n**Action Steps:** \n- **Standardize Relevance Scoring:** \n - Define clear criteria (e.g., regulatory impact, business criticality, frequency of use) to score documents objectively. \n - Use AI/ML tools (if available) to auto-tag relevance based on content analysis. \n- **Review Low-Scoring Documents (Indexes 25, 30):** \n - Determine if these documents are obsolete, redundant, or incorrectly tagged. \n - Archive or deprecate irrelevant files to reduce clutter. \n- **Flag High-Scoring Documents (Indexes 0, 49, 1):** \n - Prioritize review and updates for documents with high relevance to ensure compliance. \n\n#### **3. General Compliance Framework Enhancement** \nTo prevent recurrence, implement the following best practices: \n- **Automated Compliance Checks:** \n - Use compliance software (e.g., **OneTrust, LogicGate, Smarsh**) to flag missing/expired documents. \n- **Regular Training & Accountability:** \n - Train staff on proper document handling and compliance protocols. \n - Assign a **Compliance Officer** to oversee document integrity. \n- **Periodic Reassessment:** \n - Conduct quarterly audits to verify document accuracy, relevance, and accessibility. \n\n### **Final Recommendation Summary** \n1. **Locate & properly index missing documents.** \n2. **Adopt a DMS with strict metadata requirements.** \n3. **Standardize relevance scoring and purge obsolete files.** \n4. **Automate compliance monitoring and enforce accountability.** \n\nBy implementing these steps, the organization can resolve current compliance gaps and establish a robust, future-proof document management process. \n\nWould you like a tailored checklist for immediate execution?"
|
|
]
|
|
}
|
|
2025-04-21 22:16:43,889 - root - INFO - Document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11 processed successfully
|
|
2025-04-21 22:16:43,903 - root - INFO - Document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11 processed successfully
|
|
2025-04-21 22:16:54,195 - root - INFO - Retrieved analysis for document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11: {
|
|
"document_id": "d6c97c26-b59b-4854-bf6d-6f7f58f3dc11",
|
|
"summary": "Here are the key sections and requirements extracted from the document:\n\n### 1. **Content Types**\n - Defines the types of content within the document (e.g., XML files, relationships, themes).\n\n### 2. **Relationships**\n - Specifies the relationships between different parts of the document, such as how document parts are connected.\n\n### 3. **Document Content**\n - Contains the main content of the document, including text and formatting.\n\n### 4. **Document Relationships**\n - Defines the relationships specific to the main document part.\n\n### 5. **Theme**\n - Contains theme-related settings for the document, including color schemes and fonts.\n\n### 6. **Settings**\n - Includes document settings such as proofing options, compatibility settings, and other Word-specific configurations.\n\n### 7. **Numbering**\n - Defines numbering formats and styles used in the document.\n\n### 8. **Styles**\n - Specifies the styles applied throughout the document, including paragraph and character styles.\n\n### 9. **Web Settings**\n - Contains settings related to how the document is displayed in a web browser.\n\n### 10. **Font Table**\n - Lists the fonts used in the document and their properties.\n\n### 11. **Core Properties**\n - Includes metadata about the document, such as title, author, and creation date.\n\n### 12. **Application Properties**\n - Contains properties specific to the application (e.g., Microsoft Word) used to create the document.\n\n### Key Requirements:\n- The document must adhere to the defined content types and relationships.\n- Proper formatting and styling must be applied as specified in the styles and numbering sections.\n- Theme settings must be consistent throughout the document.\n- Metadata (core and application properties) must be accurately filled out.\n- Web settings should ensure proper display if the document is viewed online.\n\nThis summary provides an overview of the document's structure and key components, which are essential for maintaining consistency and functionality in the Word document.",
|
|
"issues": [
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 1
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 2
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "high",
|
|
"rank": 3
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 4
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 5
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "medium",
|
|
"rank": 6
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 7
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 8
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 9
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 10
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 11
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 12
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 13
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 14
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 15
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 16
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 17
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 18
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 19
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 20
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 21
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 22
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 23
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 24
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 25
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 26
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 27
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 28
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 29
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 30
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 31
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 32
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 33
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 34
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 35
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 36
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 37
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 38
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 39
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 40
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 41
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 42
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 43
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 44
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 45
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 46
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 47
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 48
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 49
|
|
},
|
|
{
|
|
"issue": null,
|
|
"severity": "low",
|
|
"rank": 50
|
|
}
|
|
],
|
|
"recommendations": [
|
|
"### **Comprehensive Compliance Recommendation for Document Issues** \n\n#### **1. Missing or Unidentified Documents (Indexes 0, 49, 1, 30, 25)** \n**Issue:** Several documents are either missing (`document=None`) or improperly indexed, leading to potential compliance risks. \n\n**Action Steps:** \n- **Conduct a Document Audit:** \n - Identify all missing documents (`index=0, 49, 1, 30, 25`) and verify if they were misplaced, incorrectly labeled, or never properly stored. \n - Cross-reference with a master document registry to confirm expected files. \n- **Implement a Document Tracking System:** \n - Use a **Document Management System (DMS)** with version control, metadata tagging, and audit trails. \n - Assign unique identifiers (e.g., `DOC-001`, `POL-2024-01`) to prevent indexing errors. \n- **Enforce Mandatory Metadata:** \n - Require fields such as: \n - Document title \n - Version number \n - Creation/modification dates \n - Author/owner \n - Compliance status (e.g., \"Reviewed,\" \"Pending Approval\") \n\n#### **2. Inconsistent Relevance Scores (Ranging from 0.23 to 0.98)** \n**Issue:** The relevance scores vary significantly, indicating potential inconsistencies in document classification, retrieval, or applicability. \n\n**Action Steps:** \n- **Standardize Relevance Scoring:** \n - Define clear criteria (e.g., regulatory impact, business criticality, frequency of use) to score documents objectively. \n - Use AI/ML tools (if available) to auto-tag relevance based on content analysis. \n- **Review Low-Scoring Documents (Indexes 25, 30):** \n - Determine if these documents are obsolete, redundant, or incorrectly tagged. \n - Archive or deprecate irrelevant files to reduce clutter. \n- **Flag High-Scoring Documents (Indexes 0, 49, 1):** \n - Prioritize review and updates for documents with high relevance to ensure compliance. \n\n#### **3. General Compliance Framework Enhancement** \nTo prevent recurrence, implement the following best practices: \n- **Automated Compliance Checks:** \n - Use compliance software (e.g., **OneTrust, LogicGate, Smarsh**) to flag missing/expired documents. \n- **Regular Training & Accountability:** \n - Train staff on proper document handling and compliance protocols. \n - Assign a **Compliance Officer** to oversee document integrity. \n- **Periodic Reassessment:** \n - Conduct quarterly audits to verify document accuracy, relevance, and accessibility. \n\n### **Final Recommendation Summary** \n1. **Locate & properly index missing documents.** \n2. **Adopt a DMS with strict metadata requirements.** \n3. **Standardize relevance scoring and purge obsolete files.** \n4. **Automate compliance monitoring and enforce accountability.** \n\nBy implementing these steps, the organization can resolve current compliance gaps and establish a robust, future-proof document management process. \n\nWould you like a tailored checklist for immediate execution?"
|
|
]
|
|
}
|
|
2025-04-21 22:18:34,156 - root - ERROR - Error retrieving analysis: list object has no element 1
|
|
2025-04-21 22:18:34,163 - root - ERROR - Traceback (most recent call last):
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\main.py", line 138, in get_analysis
|
|
return templates.TemplateResponse(
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\starlette\templating.py", line 209, in TemplateResponse
|
|
return _TemplateResponse(
|
|
^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\starlette\templating.py", line 40, in __init__
|
|
content = template.render(context)
|
|
^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\jinja2\environment.py", line 1295, in render
|
|
self.environment.handle_exception()
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\jinja2\environment.py", line 942, in handle_exception
|
|
raise rewrite_traceback_stack(source=source)
|
|
File "src\templates\analysis.html", line 1, in top-level template code
|
|
{% extends "base.html" %}
|
|
File "src\templates\base.html", line 34, in top-level template code
|
|
{% block content %}{% endblock %}
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "src\templates\analysis.html", line 50, in block 'content'
|
|
{{ analysis.recommendations[loop.index0]|markdown|safe }}
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "c:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\src\main.py", line 44, in markdown_filter
|
|
return markdown.markdown(text, extensions=['extra', 'nl2br'])
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\markdown\core.py", line 482, in markdown
|
|
return md.convert(text)
|
|
^^^^^^^^^^^^^^^^
|
|
File "C:\Users\babaw\Documents\Work\Mana Knight Digital\ds_task_scp\.venv\Lib\site-packages\markdown\core.py", line 341, in convert
|
|
if not source.strip():
|
|
^^^^^^^^^^^^
|
|
jinja2.exceptions.UndefinedError: list object has no element 1
|
|
|
|
2025-04-21 22:18:40,245 - root - INFO - Deleted uploaded file: data/uploads/d6c97c26-b59b-4854-bf6d-6f7f58f3dc11_4.Scope of Work.docx
|
|
2025-04-21 22:18:41,105 - root - INFO - Deleted document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11 from index
|
|
2025-04-21 22:18:41,105 - root - INFO - Removed document d6c97c26-b59b-4854-bf6d-6f7f58f3dc11 from vector store
|
|
2025-04-21 22:18:49,878 - root - INFO - Deleted uploaded file: data/uploads/2de0becf-fcfc-4793-aad0-7369180c6980_3.Bill of Quantities.docx
|
|
2025-04-21 22:18:50,166 - root - INFO - Deleted document 2de0becf-fcfc-4793-aad0-7369180c6980 from index
|
|
2025-04-21 22:18:50,166 - root - INFO - Removed document 2de0becf-fcfc-4793-aad0-7369180c6980 from vector store
|
|
2025-04-21 22:18:54,080 - root - INFO - Deleted uploaded file: data/uploads/3f10b972-3eb6-43c9-bfeb-777d0a33c4a3_8.form of tender.docx
|
|
2025-04-21 22:18:54,321 - root - INFO - Deleted document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3 from index
|
|
2025-04-21 22:18:54,322 - root - INFO - Removed document 3f10b972-3eb6-43c9-bfeb-777d0a33c4a3 from vector store
|
|
2025-04-21 22:18:57,881 - root - INFO - Deleted uploaded file: data/uploads/befee86e-cc92-43cf-b935-3c8ec78ef275_4.Scope of Work.docx
|
|
2025-04-21 22:18:58,105 - root - INFO - Deleted document befee86e-cc92-43cf-b935-3c8ec78ef275 from index
|
|
2025-04-21 22:18:58,105 - root - INFO - Removed document befee86e-cc92-43cf-b935-3c8ec78ef275 from vector store
|
|
2025-04-21 22:19:02,930 - root - INFO - Deleted uploaded file: data/uploads/a444141c-e0c0-4ee9-a448-4555494aaede_3.Bill of Quantities.docx
|
|
2025-04-21 22:19:03,739 - root - INFO - Deleted document a444141c-e0c0-4ee9-a448-4555494aaede from index
|
|
2025-04-21 22:19:03,748 - root - INFO - Removed document a444141c-e0c0-4ee9-a448-4555494aaede from vector store
|
|
2025-04-21 22:19:18,005 - root - INFO - Processing upload for document ID: 77063b1d-633c-421e-9591-cde2eb90a979
|
|
2025-04-21 22:19:18,008 - root - INFO - File saved to data/uploads/77063b1d-633c-421e-9591-cde2eb90a979_7.Supplier SQualification requirements.docx
|
|
2025-04-21 22:19:18,041 - root - INFO - Processing document 77063b1d-633c-421e-9591-cde2eb90a979 with content length: 15335
|
|
2025-04-21 22:19:18,695 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 22:19:19,239 - root - INFO - Stored embedding for document 77063b1d-633c-421e-9591-cde2eb90a979
|
|
2025-04-21 22:20:24,699 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 22:22:05,759 - root - INFO - Document 77063b1d-633c-421e-9591-cde2eb90a979 processed successfully
|
|
2025-04-21 22:22:05,775 - root - INFO - Document 77063b1d-633c-421e-9591-cde2eb90a979 processed successfully
|
|
2025-04-21 22:31:21,934 - root - INFO - Processing upload for document ID: e79aeb90-799a-4d06-9efd-1d19315eebcc
|
|
2025-04-21 22:31:21,936 - root - INFO - File saved to data/uploads/e79aeb90-799a-4d06-9efd-1d19315eebcc_2.Tender Specifications.docx
|
|
2025-04-21 22:31:21,953 - root - INFO - Processing document e79aeb90-799a-4d06-9efd-1d19315eebcc with content length: 17509
|
|
2025-04-21 22:32:04,265 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/embed "HTTP/1.1 200 OK"
|
|
2025-04-21 22:32:05,457 - root - INFO - Stored embedding for document e79aeb90-799a-4d06-9efd-1d19315eebcc
|
|
2025-04-21 22:33:04,556 - httpx - INFO - HTTP Request: POST https://api.cohere.com/v1/rerank "HTTP/1.1 200 OK"
|
|
2025-04-21 22:34:43,603 - root - INFO - Document e79aeb90-799a-4d06-9efd-1d19315eebcc processed successfully
|
|
2025-04-21 22:34:43,611 - root - INFO - Document e79aeb90-799a-4d06-9efd-1d19315eebcc processed successfully
|