152 lines
5.7 KiB
Plaintext
152 lines
5.7 KiB
Plaintext
|
✅ Database tables created successfully
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
PRODUCTION-READY STOCK INTELLIGENCE SYSTEM
|
||
|
|
======================================================================
|
||
|
|
Started: 2025-11-06 03:15:22
|
||
|
|
⚠️ RUNNING IN TEST MODE
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 1: EXTRACTING STOCK LISTINGS
|
||
|
|
======================================================================
|
||
|
|
📂 Loading existing listings from data/listings/all_listings_combined.json
|
||
|
|
✅ Loaded 3 stocks from file
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 2: IMPORTING TO DATABASE
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
📥 Importing listings from data/listings/all_listings_combined.json...
|
||
|
|
✅ Imported 0 stocks
|
||
|
|
|
||
|
|
⚠️ Limited to 2 stocks for testing
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 3: SCRAPING FINANCIAL DATA
|
||
|
|
======================================================================
|
||
|
|
============================================================
|
||
|
|
YAHOO FINANCE SCRAPING
|
||
|
|
============================================================
|
||
|
|
|
||
|
|
🔍 Scraping SHOP.TO...
|
||
|
|
✅ SHOP.TO data scraped successfully
|
||
|
|
|
||
|
|
🔍 Scraping AAPL...
|
||
|
|
✅ AAPL data scraped successfully
|
||
|
|
|
||
|
|
============================================================
|
||
|
|
✅ Successfully scraped: 2
|
||
|
|
❌ Failed: 0
|
||
|
|
📁 Data saved to: data/financials/
|
||
|
|
============================================================
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 4: CALCULATING FINANCIAL METRICS
|
||
|
|
======================================================================
|
||
|
|
Calculating metrics for SHOP.TO...
|
||
|
|
Calculating metrics for AAPL...
|
||
|
|
✅ Calculated metrics for 2 stocks
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 5: SCRAPING NEWS & PRESS RELEASES
|
||
|
|
======================================================================
|
||
|
|
📡 Using SerpAPI for robust news collection...
|
||
|
|
======================================================================
|
||
|
|
SERPAPI NEWS & PRESS RELEASE SCRAPER
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
🔍 Fetching news & PR via SerpAPI for SHOP.TO - Shopify Inc.
|
||
|
|
Searching Google News via SerpAPI: "Shopify Inc." OR "SHOP.TO" stock earnings financial...
|
||
|
|
Found 55 articles
|
||
|
|
Searching press release sites via SerpAPI...
|
||
|
|
Found 10 press releases
|
||
|
|
✅ Saved SerpAPI data: 55 news, 10 PR
|
||
|
|
|
||
|
|
🔍 Fetching news & PR via SerpAPI for AAPL - Apple Inc.
|
||
|
|
Searching Google News via SerpAPI: "Apple Inc." OR "AAPL" stock earnings financial...
|
||
|
|
Found 55 articles
|
||
|
|
Searching press release sites via SerpAPI...
|
||
|
|
Found 10 press releases
|
||
|
|
✅ Saved SerpAPI data: 55 news, 10 PR
|
||
|
|
|
||
|
|
✅ Completed scraping 2 stocks via SerpAPI
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 6: SCRAPING SEC EDGAR FILINGS
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
🔍 Scraping SEC filings for AAPL...
|
||
|
|
Found CIK: 0000320193
|
||
|
|
Found 100 recent filings
|
||
|
|
Scraping 10-K from 2025-10-31...
|
||
|
|
Scraping 8-K from 2025-10-30...
|
||
|
|
Scraping 10-Q from 2025-08-01...
|
||
|
|
Scraping DEF 14A from 2025-01-10...
|
||
|
|
Getting ownership data...
|
||
|
|
✅ Saved SEC data to data/sec_filings/AAPL_sec_filings.json
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 7: SCRAPING SEDAR+ FILINGS
|
||
|
|
======================================================================
|
||
|
|
======================================================================
|
||
|
|
SEDAR+ SCRAPER
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
SCRAPING SEDAR+ FOR: SHOP.TO - Shopify Inc.
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
🔍 Searching SEDAR+ for Shopify Inc. (SHOP.TO)...
|
||
|
|
Saved search results to data/sedar_filings/SHOP.TO_sedar_search.html
|
||
|
|
✅ Found 0 potential filings
|
||
|
|
✅ Saved SEDAR+ data to data/sedar_filings/SHOP.TO_sedar_data.json
|
||
|
|
|
||
|
|
✅ Completed scraping 1 companies
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 8: GENERATING REPORTS
|
||
|
|
======================================================================
|
||
|
|
✅ Generated 23 comprehensive reports
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
STEP 9: EXPORTING TO CSV
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
CSV EXPORT - ALL DATA
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
📤 Exporting stock list to data/exports/stocks_export.csv...
|
||
|
|
✅ Exported 23 stocks to CSV
|
||
|
|
|
||
|
|
📤 Exporting detailed financials to data/exports/stocks_detailed.csv...
|
||
|
|
⚠️ No financial metrics found in database
|
||
|
|
|
||
|
|
📤 Exporting news summary to data/exports/news_summary.csv...
|
||
|
|
✅ Exported 0 news articles
|
||
|
|
|
||
|
|
📤 Exporting filings summary to data/exports/filings_summary.csv...
|
||
|
|
✅ Exported 0 filings
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
✅ Created 3 CSV files:
|
||
|
|
- data/exports/stocks_export.csv
|
||
|
|
- data/exports/news_summary.csv
|
||
|
|
- data/exports/filings_summary.csv
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
======================================================================
|
||
|
|
FINAL STATISTICS
|
||
|
|
======================================================================
|
||
|
|
Duration: 0:02:10.036167
|
||
|
|
Stocks processed: 3
|
||
|
|
Financials scraped: 2
|
||
|
|
Metrics calculated: 2
|
||
|
|
News articles collected: 2
|
||
|
|
Filings scraped: 2
|
||
|
|
Errors: 0
|
||
|
|
======================================================================
|
||
|
|
|
||
|
|
✅ PIPELINE COMPLETED SUCCESSFULLY!
|