What is businesswire-scraper?

businesswire-scraper is a Python tool that monitors businesswire.com press releases continuously, capturing every new release that matches a set of keywords or ticker symbols you configure. Instead of scraping the website's search interface directly, since that is blocked by anti-bot protection, it uses two officially published feeds listed in the site's own robots.txt file. A live feed is polled every fifteen minutes to catch new releases as they appear, and a daily archive hosted on public cloud storage provides a backfill safety net for anything the live window might have missed. When a new release matches one of your configured search terms, using case-insensitive substring matching against the title and description, the scraper needs to download the full HTML of that release page. Those pages are also anti-bot protected, so the tool uses a two-stage approach: first, a specially patched browser called Patchright opens the site and simulates real user behavior to collect valid session cookies, then those cookies are handed off to a lightweight HTTP client called curl_cffi, which downloads all newly matched release pages in parallel. The browser and HTTP client are configured to impersonate the same browser version, which the README says is required to avoid triggering a mismatch check in the anti-bot system. Everything is stored locally: HTML files and JSON metadata are organized into dated folders on disk, and a SQLite database handles deduplication so repeated passes never save the same release twice. A cron job can run the whole pass every fifteen minutes without collisions. Typical runtime is 30 to 60 seconds for a batch of up to ten new releases. The tool depends on a paid residential proxy service, whose credentials go into a gitignored configuration file.