What is scraping-skills?

This repository is a collection of "skills" for AI coding agents like Claude. A skill here means a markdown file that teaches the agent how to approach a specific type of web scraping task. The idea is that when you ask your AI assistant to extract data from a website, the relevant skill gives the agent a structured way to think about the problem, including whether to build a scraper from scratch or use an existing paid service. The skills are organized into two groups. The first group covers the engineering side of scraping. One skill helps the agent decide between building and buying, walking through techniques from simple HTTP requests to stealth browsers. Another focuses on bypassing anti-bot systems like Cloudflare and DataDome. A third helps package a scraper as a deployable product on the Apify platform. The second group covers specific domains, routing the agent to the right approach for jobs, e-commerce, reviews, social media, leads, real estate, travel, food delivery, and SEO data. Thirdwatch, the company maintaining this repository, runs over 70 production scrapers on the Apify Store. When a maintained scraper already exists for a target site, the skill points the agent to it so the user does not have to deal with anti-bot defenses. When no ready-made scraper exists, the engineering skills guide the agent through building one. The Apify scrapers use pay-per-result pricing, meaning you are billed per row of data returned, with a free tier available. To install a skill, you copy its folder into your project's skills directory. The skill activates automatically when your request matches its description. The same skill files work with any agent runtime that supports the Agent Skills format, not just Claude. The README includes a section on responsible use. The skills target publicly accessible data, and users are expected to respect each site's Terms of Service, robots.txt files, and privacy laws like GDPR and CCPA. The project is released under the MIT license.