get-set, Fetch! icon
get-set, Fetch! icon

get-set, Fetch!

Nodejs web scraper. Use it from your own code, via command line or docker container. Supports multiple storage options: SQLite, MySQL, PostgreSQL. Supports multiple browser or dom-like clients: Puppeteer, Playwright, Cheerio, JSdom.

get-set, Fetch! screenshot 1

Cost / License

  • Free
  • Open Source (MIT)

Application type

Platforms

  • Mac
  • Windows
  • Linux
Alternatives  
0likes
24alternatives
0articles

Features

  1.  Command line interface
  2. Docker icon  Support for Docker

 Tags

get-set, Fetch! News & Activities

Highlights All activities

Recent activities

No activities found.

get-set, Fetch! information

AlternativeTo Category

OS & Utilities

GitHub repository

  •  113 Stars
  •  18 Forks
  •  12 Open Issues
  •   Updated  
View on GitHub

Popular alternatives

View all
get-set, Fetch! was added to AlternativeTo by asabau on and this page was last updated .
No comments or reviews, maybe you want to be first?

Official Links

AppStores & Other Links

What is get-set, Fetch!?

get-set, Fetch! is a plugin based, nodejs web scraper. It scrapes, stores and exports data. At its core, an ordered list of plugins (default or custom defined) is executed against each to be scraped web resource.

Supports multiple storage options: SQLite, MySQL, PostgreSQL. Supports multiple browser or dom-like clients: Puppeteer, Playwright, Cheerio, Jsdom.

For quick, small projects under 10K URLs storing the queue and scraped content under SQLite is fine. For anything larger use PostgreSQL. You will be able to start/stop/resume the scraping process across multiple scraper instances each with its own IP and/or dedicated proxies. Using PostgreSQL it takes 90 minutes to scrape 1 million URLs with a concurrency of 100 parallel scrape actions. That's 5.5ms per scraped URL.