Skip to content
BeClaude

scraping-skills

New
2GitHub TrendingGeneralby thirdwatch-dev

Web scraping skills for Claude & coding agents — anti-bot bypass, build-vs-buy, and ready-made scrapers for jobs, e-commerce, reviews, social, leads, real estate, travel, food & SEO. npx skills add thirdwatch-dev/scraping-skills

Summary

This skill equips Claude and coding agents with web scraping expertise, covering anti-bot bypass techniques, build-vs-buy decision frameworks, and ready-made scrapers for domains like jobs, e-commerce, reviews, social media, leads, real estate, travel, food, and SEO.

  • It helps developers efficiently extract structured data from websites while avoiding common pitfalls like IP blocking and CAPTCHAs.

Install & Usage

1
Create the skills directory
mkdir -p .claude/skills
2
Download the skill file
mkdir -p .claude/skills && curl -o .claude/skills/scraping-skills.md https://raw.githubusercontent.com/thirdwatch-dev/scraping-skills/main/SKILL.md
3
Invoke in Claude Code
/scraping-skills

Use Cases

Scrape job listings from multiple boards to aggregate opportunities for a job search tool.
Extract product prices and availability from e-commerce sites for competitive analysis.
Collect customer reviews from review platforms to perform sentiment analysis.
Gather social media posts or profiles for lead generation or market research.
Pull real estate property details and pricing from listing sites for a market report.
Scrape restaurant menus and delivery options from food platforms for a local guide.

Usage Examples

1

/scraping-skills Help me scrape job postings from Indeed for 'data scientist' in New York, avoiding rate limits.

2

/scraping-skills I need to extract product prices from Amazon and Walmart for price comparison. Which approach should I use?

3

/scraping-skills Show me how to build a scraper for Yelp reviews that bypasses anti-bot measures.

View source on GitHub
code-reviewapiagent

Security Audits

LicenseUnknownSourceWarnRepositoryPass

Frequently Asked Questions

What is scraping-skills?

This skill equips Claude and coding agents with web scraping expertise, covering anti-bot bypass techniques, build-vs-buy decision frameworks, and ready-made scrapers for domains like jobs, e-commerce, reviews, social media, leads, real estate, travel, food, and SEO. It helps developers efficiently extract structured data from websites while avoiding common pitfalls like IP blocking and CAPTCHAs.

How to install scraping-skills?

To install scraping-skills: create the skills directory (mkdir -p .claude/skills), then run: mkdir -p .claude/skills && curl -o .claude/skills/scraping-skills.md https://raw.githubusercontent.com/thirdwatch-dev/scraping-skills/main/SKILL.md. Finally, /scraping-skills in Claude Code.

What is scraping-skills best for?

scraping-skills is a skill categorized under General. It is designed for: code-review, api, agent. Created by thirdwatch-dev.

What can I use scraping-skills for?

scraping-skills is useful for: Scrape job listings from multiple boards to aggregate opportunities for a job search tool.; Extract product prices and availability from e-commerce sites for competitive analysis.; Collect customer reviews from review platforms to perform sentiment analysis.; Gather social media posts or profiles for lead generation or market research.; Pull real estate property details and pricing from listing sites for a market report.; Scrape restaurant menus and delivery options from food platforms for a local guide..