🌐 Web & Scraping

Resilient Scraper

Anti-block web scraping with automatic fallback chain. Never lose data to a 403 again.

What it does

Resilient Scraper wraps every fetch request in an intelligent fallback chain: simple fetch → browser automation → proxy → cached copy. When a site blocks your first attempt, it automatically escalates to the next method without you writing a single line of retry logic.

It rotates user agents per domain, tracks success rates, and remembers which approach works best for each site. Rate limits and CAPTCHAs are handled gracefully with configurable backoff. If all live attempts fail, it serves the last known cached version.

Use cases

• Research agents that need reliable data from sites that block scrapers
• Monitoring pipelines where a blocked fetch means missed data
• Content aggregators pulling from multiple sources daily
• Any workflow where web_fetch fails with 403/429/timeout

Requirements

• OpenClaw 3.20+
• Python 3.10+
• Optional: Proxy credentials for advanced fallback

How to install

After purchase you’ll receive a download link by email. Extract and copy the resilient-scraper/ folder into your OpenClaw skills/ directory, then restart OpenClaw.