Commit Graph

24 Commits

Author SHA1 Message Date
Nicolas 7e002a8b06 Nick: bull mq 2024-07-30 13:27:23 -04:00
Nicolas e5b797549e Merge branch 'main' into feat/scrape-monitoring 2024-07-25 16:21:02 -04:00
rafaelsideguide 309728a482 updated logs 2024-07-25 09:48:06 -03:00
Gergo Moricz 7cd9bf92e3 feat: scrape event logging to DB 2024-07-24 14:31:25 +02:00
rafaelsideguide 4381109dd8 added default values and fixed pdf bug 2024-06-26 09:00:54 -03:00
Rafael Miller 3e2e76311c Merge branch 'main' into feat/issue-205 2024-06-14 11:25:20 -03:00
rafaelsideguide 676d6e8ab5 Added pageOptions.removeTags 2024-06-13 10:51:05 -03:00
rafaelsideguide e37d151404 added parsePDF option to pageOptions
user can decide if they are going to let us take care of the parse or they are going to parse the pdf by themselves
2024-06-12 15:06:47 -03:00
rafaelsideguide 01c9f071fa fixed 2024-06-12 11:27:06 -03:00
rafaelsideguide ee282c3d55 Added allowBackwardCrawling option 2024-06-11 15:24:39 -03:00
rafaelsideguide 184e4678f1 bugfix on idempotency key check 2024-05-23 11:47:04 -03:00
rafaelsideguide 3f460af6c5 Added idempotency key to crawl route 2024-05-07 15:29:27 -03:00
Nicolas bdbee963f7 Merge branch 'main' into nsc/cancel-job 2024-05-07 10:13:43 -07:00
rafaelsideguide e1f52c538f nested includeHtml inside pageOptions 2024-05-07 13:40:24 -03:00
Nicolas 6d5da358cc Nick: cancel job 2024-05-06 17:16:43 -07:00
rafaelsideguide 509250c4ef changed to includeHtml 2024-05-06 19:45:56 -03:00
rafaelsideguide 538355f1af Added toMarkdown option 2024-05-06 11:36:44 -03:00
Nicolas f3c190c21c Nick: 2024-04-23 16:47:24 -07:00
rafaelsideguide 849c0b6ebf [Feat] Added blocklist for social media urls 2024-04-23 18:50:35 -03:00
Nicolas 898d729a84 Nick: tests 2024-04-21 11:27:31 -07:00
Nicolas 5cdbf3a0ac Nick: cleaner functions to handle authenticated requests that dont require ifs everywhere 2024-04-21 10:36:48 -07:00
Caleb Peffer be75aaa195 Caleb: first version of supabase proxy to make db authentication optional 2024-04-21 09:31:22 -07:00
Nicolas 0db0874b00 Nick: 2024-04-20 19:37:45 -07:00
Nicolas 23b2190e5d Nick: 2024-04-20 16:38:05 -07:00