Scrape websites that block the standard Python requests library by mimicking a real Chrome or Firefox browser connection signature.
Make asynchronous HTTP/2 requests from Python without the standard library limitations, requests and httpx cannot do HTTP/2.
Connect to WebSocket endpoints from a Python script while appearing as a real browser to the server.
Rotate proxies on a per-request basis in a scraping pipeline while maintaining realistic browser fingerprints throughout.
| lexiforest/curl_cffi | coleam00/ottomator-agents | adamchainz/django-cors-headers | |
|---|---|---|---|
| Stars | 5,584 | 5,584 | 5,586 |
| Language | Python | Python | Python |
| Setup difficulty | easy | moderate | easy |
| Complexity | 2/5 | 3/5 | 1/5 |
| Audience | developer | developer | developer |
Figures from each repo's GitHub metadata at analysis time.
Install with a single pip command, no compilation step needed on Linux, macOS, or Windows.
curl_cffi is a Python library for making HTTP requests that can disguise itself as a real browser. Websites use various technical signatures to detect automated scripts, and curl_cffi defeats those checks by mimicking the exact connection patterns that Chrome, Firefox, Safari, and other browsers produce. If your scraper or automation tool keeps getting blocked without a clear reason, that blocking is often based on these signatures rather than your IP address, and this library addresses that. The library wraps a modified version of curl called curl-impersonate. When you make a request with the impersonate option set to a specific browser, the library reproduces that browser's TLS handshake pattern (sometimes called a JA3 fingerprint) and its HTTP/2 negotiation behavior. These are low-level connection details that other Python HTTP libraries like requests or httpx cannot replicate because they use their own TLS stacks. Beyond fingerprint matching, curl_cffi supports modern HTTP versions including HTTP/2 (which the standard requests library does not support) and HTTP/3. It works in both synchronous and asynchronous code, supports WebSocket connections, handles proxy rotation per request, and includes built-in retry logic. Its API is intentionally similar to the requests library so existing code is easy to adapt. A command-line tool called curl-cffi ships with the library for quick manual testing. Installation is a single pip command and works on Linux, macOS, and Windows without any compilation step. The library is MIT licensed. Commercial support, additional browser fingerprints, and integrated anti-bot bypass solutions are available through a separate paid service called impersonate.pro.
A Python HTTP library that makes your scraper or automation script look like a real Chrome, Firefox, or Safari browser by copying their exact connection signatures, bypassing most bot-detection blocks.
Mainly Python. The stack also includes Python, C.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice.
Setup difficulty is rated easy, with roughly 5min to a first successful run.
Mainly developer.
This repo across BitVibe Labs
Verify against the repo before relying on details.