Gergő Móricz 350d00d27a fix(crawler): treat XML files as sitemaps (temporarily)

2024-11-15 20:09:20 +01:00

..

__tests__

WebScraper refactor into scrapeURL (#714)

2024-11-07 20:57:33 +01:00

custom

WebScraper refactor into scrapeURL (#714)

2024-11-07 20:57:33 +01:00

utils

fix(crawl/maxDepth): fix maxDepth behaviour

2024-11-11 22:02:17 +01:00

crawler.ts

fix(crawler): treat XML files as sitemaps (temporarily)

2024-11-15 20:09:20 +01:00

sitemap.ts

fix(sitemap): scrape with tlsclient

2024-11-14 18:56:32 +01:00