created a data ingestion module.

This commit is contained in:
timothyafolami
2024-08-07 18:03:15 +01:00
parent 8e6acc7cf8
commit 228fffefd8
7 changed files with 43 additions and 3 deletions
+9
View File
@@ -29,3 +29,12 @@
2024-08-07 17:49:19,962 - INFO - Receiving the search query
2024-08-07 17:49:29,498 - INFO - Searching for what is lda?
2024-08-07 17:49:29,876 - INFO - Search completed
2024-08-07 18:01:11,466 - INFO - Loading the embeddings
2024-08-07 18:01:11,468 - INFO - Load pretrained SentenceTransformer: BAAI/bge-small-en
2024-08-07 18:01:14,748 - INFO - Embeddings loaded
2024-08-07 18:01:14,748 - INFO - Loading data from ./data
2024-08-07 18:01:58,703 - INFO - Data loaded
2024-08-07 18:01:58,703 - INFO - Creating vector store
2024-08-07 18:02:08,752 - INFO - Vector store created
2024-08-07 18:02:08,752 - INFO - Saving the vector store
2024-08-07 18:02:08,752 - INFO - Vector store saved