ai-crawler repositories

Trending repositories tagged ai-crawler, ranked by trending score. Updated daily.

  1. 1
    firecrawl/firecrawl

    Knowledge infrastructure to make your agents smarter. Search, scrape and access 550+ data providers through one API.

    GitHub · TypeScript · 205,629 stars · 10,754 forks · +2,082 stars in 24 hours · Trending score 5.92

  2. 2
    ScrapeGraphAI/Scrapegraph-ai

    Python scraper based on AI

    GitHub · Python · 32,236 stars · 3,302 forks · 0 stars in 24 hours · Trending score 0.00

  3. 3
    puneetindersingh/open-seo-crawler

    Free SEO crawler, website audit and AI crawler checker. Self-hosted and open source: crawls a site for technical SEO issues and checks that Googlebot, Bingbot and AI crawlers (GPTBot, PerplexityBot, OAI-SearchBot) get the same readable page. CMS-aware, XLSX export, free Screaming Frog alternative.

    GitHub · Python · 65 stars · 20 forks · 0 stars in 24 hours · Trending score 0.00

  4. 4
    makralabs/makra

    Makra is a memory layer between the web and your AI agents. It serves real-time structured data from the open web at the cost of a vector search.

    GitHub · Python · 30 stars · 0 forks · 0 stars in 24 hours · Trending score 0.00