doc-scraper

Sriram-PR/doc-scraper

Web crawler that converts documentation sites to clean Markdown and JSONL for LLM ingestion (RAG, training data).

  • data-preparation
  • llm
  • web-scraper
  • documentation
  • golang-cli
  • mcp
  • mcp-server
  • web-crawler

More in Scrapers

collyby asciimoo

Fast and Elegant Scraping Framework for Gophers.

github.com/gocolly/colly/v2Go 1.24+ 12 dependenciesv2.2.0updated 17d ago

25.6k

GoQueryby PuerkitoBio

GoQuery brings a syntax and a set of features similar to jQuery to the Go language.

github.com/PuerkitoBio/goqueryGo 1.25+ 2 dependenciesv1.13.0updated today

15k

xurlsby mvdan

Extract urls from text.

mvdan.cc/xurls/v2Go 1.26+ 2 dependenciesv2.6.0updated today

1.3k

dataflowkitby slotix

Web scraping Framework to turn websites into structured data.

github.com/slotix/dataflowkitGo 1.13+ 16 dependenciesupdated 3y ago

714

pagserby foolin

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler.

github.com/foolin/pagserGo 1.21+ 4 dependenciesv0.1.6updated 3y ago

112

Tagifyby zoomio

Produces a set of tags from given source.

github.com/zoomio/tagifyGo 1.22+ 7 dependenciesv0.62.0updated 2y ago

39

go-recipeby kkyr

A package for scraping recipes from websites.

github.com/kkyr/go-recipeGo 1.20+ 4 dependenciesv0.4.0updated 3y ago

33

walkerby cyucelen

Seamlessly fetch paginated data from any source. Simple and high performance API scraping included.

github.com/cyucelen/walkerGo 1.18+ 4 dependenciesv0.1.0updated 3y ago

16
All 10 in Scrapers
Esc

Type to search every entry. ↑↓ to move, Enter to open.