--- source_url: "https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection" title: How to Bypass Anti-Bot Protection in 2026 mirrored_at: 2026-08-04T12:31:46.073Z host: scrapfly.io cited_in_42a: true mirror_canonical: "https://index.42a.ai/scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection" --- > **Original source:** https://scrapfly.io/blog/posts/how-to-bypass-anti-bot-protection You hit a 403. Before you write any bypass code, you need one thing, the vendor name. A bypass that works on Cloudflare fails on Kasada. A session approach that clears PerimeterX doesn't work on Akamai. Every vendor runs a different detection model, and building for the wrong one wastes the effort. In this guide, you'll learn to identify which vendor you're facing from raw HTTP response headers, cookie names, and JS file signatures. Then you'll see how each system detects bots and how to bypass each of the eight major vendors with working Python code. Let's get started. ## Key Takeaways - Identify the vendor before writing any bypass code. `CF-RAY` in the response header means Cloudflare; `_abck` means Akamai; a bare 429 with no body means Kasada. - No bypass technique works across all eight vendors. DataDome uses ML scoring, Kasada uses active environment interrogation, and Akamai validates behavior telemetry server-side. - Kasada is the only vendor where DIY isn't viable in production. Anti-deobfuscation protections and per-site challenge rotation break DIY solutions within days of a challenge update. - Signal consistency matters across all eight vendors. Pair a Chrome User-Agent with a Python TLS fingerprint and every vendor on this list flags you as a bot. - DIY bypass works at small scale with one vendor but breaks with each challenge update. At production scale, residential proxy costs alone often exceed a scraping API, before counting engineering hours. - Scrapfly's [Anti-Scraping Protection](https://scrapfly.io/docs/scrape-api/anti-scraping-protection) handles all eight vendors with one `asp=True` parameter; success rates range from 94% against Kasada to 98% against Cloudflare. For multi-vendor scraping at scale, it removes the per-vendor bypass maintenance from your stack. **Get web scraping tips in your inbox**Trusted by 100K+ developers and 30K+ enterprises. Unsubscribe anytime. ## How to Identify Which Anti-Bot System You're Hitting The fastest way to identify a vendor is to read the HTTP response before you attempt any bypass. You don't need to solve the challenge. You need to recognize the signature. For a deeper look at detection patterns, see [our identification guide](https://scrapfly.io/blog/posts/how-to-know-what-anti-bot-website-uses). Here's the quick reference. ### Response Headers and Status Codes Send a plain `requests.get()` with a browser User-Agent and look at the response headers right away. - **Cloudflare**: `CF-RAY` header is always present; `server: cloudflare` in the response. - **DataDome**: `X-DataDome-*` headers appear on 403 responses. - **Akamai**: `akamai-grn` reference number appears on block pages. - **Kasada**: No branded headers. You'll get a minimal 429 or 403 with almost no body. A 403 with no vendor signature usually means F5/Shape or AWS WAF. A 429 with no body is Kasada. ### Cookie Name Patterns Check the cookies set by the site before any JavaScript runs. - **Cloudflare**: `cf_clearance` (session clearance), `__cf_bm` (bot management score) - **DataDome**: `datadome`, `_dd_s` - **PerimeterX**: `_px3`, `_pxvid`, `_pxhd` - **Kasada**: `KP_UIDz` cookie; `x-kpsdk-ct` / `x-kpsdk-cd` headers - **Akamai**: `_abck` (encodes behavior telemetry) - **Incapsula**: `incap_ses_*`, `visid_incap_*`, `reese84` - **AWS WAF**: `aws-waf-token` - **F5/Shape**: `TS*`\-prefixed cookies (e.g., `TS01a2b3c4`) ### Script Tag and JS File Signatures Look at the JS files the page loads. These are consistent across sites using the same vendor. - **Cloudflare**: `challenge.js` - **DataDome**: `tags.js` - **PerimeterX**: `px.js` or `d.js` - **Kasada**: `ips.js` or `d.js` - **Akamai**: akamai-bm-telemetry scripts - **Incapsula**: Imperva JS challenge scripts - **AWS WAF**: `/challenge.js` telemetry endpoint ### Block Page Visual Signatures Most vendors brand their block pages. Cloudflare shows a branded interstitial with a Ray ID. DataDome shows a labeled CAPTCHA page. Incapsula shows an Imperva-branded challenge. AWS WAF shows a branded CAPTCHA. PerimeterX shows HUMAN Security branding. Kasada is the exception. It returns a bare 429 with no copy, no logo, and no interstitial. ### Identification Reference Table Vendor Cookies JS Signature Response Header Block Page Cloudflare `cf_clearance`, `__cf_bm` `challenge.js` `CF-RAY`, `server: cloudflare` Branded interstitial + Ray ID DataDome `datadome`, `_dd_s` `tags.js` `X-DataDome-*` Labeled CAPTCHA page PerimeterX `_px3`, `_pxvid`, `_pxhd` `px.js` / `d.js` None distinctive HUMAN Security branding Kasada `KP_UIDz`, `x-kpsdk-ct` header `ips.js` / `p.js` None Bare 429 / 403, no body Akamai `_abck` akamai-bm-telemetry `akamai-grn` Reference number page Incapsula `incap_ses_*`, `visid_incap_*`, `reese84` Imperva JS challenge None distinctive Imperva-branded challenge AWS WAF `aws-waf-token` `/challenge.js` None distinctive AWS-branded CAPTCHA F5/Shape `TS*`\-prefix (e.g., `TS01xxxxxx`) Shape AI JS Shape/F5 error headers Generic "Access Denied" Use this Python snippet to pull the raw headers and cookies before attempting any bypass: python ``` import requests url = "https://web-scraping.dev/antibot/easy" response = requests.get( url, headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"} ) print("Status:", response.status_code) print("\nHeaders:") for key, value in response.headers.items(): print(f" {key}: {value}") print("\nCookies:") for name, value in response.cookies.items(): print(f" {name}: {value[:60]}") ``` Cross-reference the output against the table above. Once you know the vendor, go to the relevant section below. ## How These Systems Detect Scrapers: Common Signals Every anti-bot vendor checks a different combination of signals, but the underlying detection layers are the same. Understanding the stack tells you why copying a User-Agent string alone doesn't work. Detection Layer What's Checked Which Vendors Use It TLS / JA3 fingerprint Cipher suite order, TLS extensions, GREASE values Cloudflare, PerimeterX, Kasada, F5, DataDome IP reputation ASN, datacenter vs. residential, known proxy ranges All eight vendors HTTP header ordering Header name order, casing, pseudo-header priority in HTTP/2 Cloudflare, PerimeterX, Akamai, F5 JS fingerprinting Browser APIs, canvas, WebGL, hardware sensors Kasada, DataDome, Akamai, PerimeterX Behavior biometrics Mouse path, scroll velocity, typing patterns DataDome, PerimeterX, Kasada, Akamai Detection happens in two stages. Before the vendor returns any HTML, it checks TLS handshake data, IP reputation scores, and HTTP header ordering. If those pass, JavaScript runs client-side and checks browser APIs, canvas rendering, hardware sensor data, and user behavior. ### The Consistency Requirement Signal consistency is non-negotiable. Pair a Chrome User-Agent with a Python TLS fingerprint and all eight vendors flag you as a bot. Your [header configuration](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-headers) must match the browser you claim to be. Your [TLS fingerprint](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-tls) must too. When you add [browser fingerprint impersonation](https://scrapfly.io/blog/posts/bypass-proxy-detection-with-browser-fingerprint-impersonation), every signal layer must agree. The vendor-specific guides below cover each signal in depth. ## Bypass by Vendor ### DataDome **Fingerprints**: `datadome` cookie, `_dd_s` session cookie, `tags.js` challenge script. DataDome uses real-time ML scoring per request rather than rule-based filtering. It trains on global traffic patterns and adapts faster than most other vendors when bypass techniques circulate publicly. Detection signals: - Real-time ML trust score per request; there's no static rule to work around - Slider CAPTCHA collects mouse path, timing, and pressure patterns during human verification - Bot/human ratio analysis across all global traffic; each request gets an independent score You won't get far with a plain `requests` session. DataDome requires a consistent browser fingerprint and behavior signals that match a real browser session. Scrapfly's ASP handles this automatically: python ``` from scrapfly import ScrapflyClient, ScrapeConfig client = ScrapflyClient(key="YOUR_SCRAPFLY_KEY") result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) print(result.scrape_result["content"]) ``` For a complete DataDome bypass tutorial: [How to Bypass DataDome Anti-Scraping](https://scrapfly.io/blog/posts/how-to-bypass-datadome-anti-scraping) ### PerimeterX / HUMAN Security **Fingerprints**: `_px3` clearance cookie, `px.js` challenge script, HUMAN Security branding on block pages. PerimeterX runs a two-stage model: backend IP scoring first, then a client-side JS challenge if your IP passes. Each customer site gets a separate ML model, so a bypass that works on one site often fails on another. Detection signals: - Backend IP scoring happens before the page loads; the vendor blocks datacenter IPs before the JS challenge runs - `px.js` POSTs behavior telemetry to `/api/v2/collector`; replaying these requests without real browser execution is detectable - "Press & Hold" human challenge collects behavior biometrics including timing and pressure patterns python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) ``` For a complete tutorial including session management: ### Kasada **Fingerprints**: `ips.js` / `p.js` challenge script, `KP_UIDz` cookie, `x-kpsdk-ct` header, encrypted rotating payloads. Kasada is the hardest vendor on this list to bypass without managed tooling. Unlike passive fingerprinting systems, Kasada's interrogation model actively probes the execution environment. Anti-debugging and anti-deobfuscation techniques detect any attempt to reverse the challenge code. A raw Kasada block looks like this: no branded page, no copy, a bare response: http ``` HTTP/2 429 content-length: 0 ``` Or sometimes a minimal 403 with a short JSON body: json ``` {"code": "forbidden", "message": ""} ``` There's no vendor branding, no Kasada logo, and no interstitial. This is what makes Kasada hard to identify: you're looking for the absence of a block page, not its presence. Detection signals: - Interrogation model: actively probes the environment rather than passively fingerprinting - Behavior biometrics: mouse path, scroll velocity, hardware sensor data - Anti-debugging: obfuscated eval chains detect any attempt to deobfuscate or reverse the challenge - Per-site challenge rotation: challenges change per deployment, so bypass code needs site-specific maintenance DIY is not viable for production use against Kasada. Challenge rotation and anti-deobfuscation protections mean any DIY solution breaks within days of a challenge update. python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # ~94% success rate; managed bypass is the only production-viable path ``` For Kasada-specific details: ### Cloudflare Bot Management **Fingerprints**: `cf_clearance` session cookie, `__cf_bm` bot management score cookie, `CF-RAY` response header. Cloudflare is the most common anti-bot system and has the most public bypass tooling. The key distinction most developers miss: Bot Management and Turnstile are two different products. Bot Management is the enterprise-grade system; Turnstile is the user-facing CAPTCHA widget. Detection signals: - TLS/JA3 fingerprint and HTTP/2 SETTINGS frame must match the claimed browser; User-Agent, Sec-CH-UA, and TLS must all be consistent - Cloudflare blocks datacenter IP ASNs before serving the JS challenge; you need residential proxies - Proof-of-work JS challenge lifecycle requires real browser execution to solve Scrapfly achieves 98% success against Cloudflare, the highest of all eight vendors: python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # 98% success rate against Cloudflare Bot Management ``` For the full Cloudflare bypass approach: [How to Bypass Cloudflare Anti-Scraping](https://scrapfly.io/blog/posts/how-to-bypass-cloudflare-anti-scraping). If you're using FlareSolverr, it struggles with Bot Fight Mode. The [curl\_cffi-based approach](https://scrapfly.io/blog/posts/how-to-bypass-cloudflare-with-flaresolverr) is more dependable. For requests-based libraries, compare [Cloudscraper alternatives](https://scrapfly.io/blog/posts/what-is-cloudscraper-and-new-alternatives). ### F5 BIG-IP / Shape Security **Fingerprints**: `TS*`\-prefixed cookies (e.g., `TS01a2b3c4`), HTTP/2 SETTINGS frame, Shape AI JS challenge. F5 BIG-IP sits in the network traffic path, not at the application layer. Shape Security (acquired by F5 in 2020) provides the bot protection layer within BIG-IP. Because F5 deploys at the network level, its fingerprinting goes particularly deep. It sees the full connection profile before your request reaches the application. Detection signals: - Network-level deployment inspects connection fingerprints before the application layer does - One of the largest IP and ASN reputation databases in the industry; datacenter ranges are almost universally blocked - TLS and HTTP/2 fingerprinting comparable in depth to PerimeterX - Shape AI JS challenge runs active probing similar to Kasada's interrogation model For F5, the right [residential proxy selection](https://scrapfly.io/blog/posts/introduction-to-proxies-in-web-scraping) matters more than for any other vendor on this list. [IP-based detection](https://scrapfly.io/blog/posts/how-to-avoid-web-scraping-blocking-ip-addresses) covers which proxy ranges to avoid at the ASN level. Use [rotating residential proxies](https://scrapfly.io/blog/posts/how-to-rotate-proxies-in-web-scraping) to avoid building a detectable request pattern. python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # 95% success rate ``` No dedicated F5 bypass guide exists yet. See the [Scrapfly F5 bypass page](https://scrapfly.io/bypass/f5) for current approach notes. ### Akamai Bot Manager **Fingerprints**: `_abck` cookie, akamai-bm-telemetry scripts, `akamai-grn` reference header on block pages. The `_abck` cookie is Akamai's fingerprint of user behavior. Its value encodes behavior telemetry collected by the JS sensor script. Replaying a valid-looking `_abck` value requires reproducing the underlying telemetry, because Akamai validates the cookie server-side on every request. Detection signals: - `_abck` validation is server-side; copying a valid cookie from a real browser session and reusing it doesn't work - Sensor data collection requires real browser execution; the JS must run in an actual browser environment to produce valid telemetry python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # 97% success rate ``` For full implementation details: [our Akamai bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-akamai-anti-scraping). For a worked marketplace example, see how to [scrape AutoScout24](https://scrapfly.io/blog/posts/how-to-scrape-autoscout24). ### Incapsula / Imperva **Fingerprints**: `reese84` JS challenge, `incap_ses_*` session cookie, `visid_incap_*` visitor ID. The `reese84` challenge combines proof-of-work with browser fingerprinting. You must solve it before you can access any page content. The resulting cookies are time-limited. Incapsula re-challenges on expiry, so you can't reuse sessions indefinitely. Detection signals: - `reese84` combines proof-of-work with fingerprinting; you must solve it, not copy it from another session - Session cookies expire and fresh challenges trigger on each new session python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # 96% success rate ``` For full implementation details: [our Incapsula / Imperva bypass guide](https://scrapfly.io/blog/posts/how-to-bypass-imperva-incapsula-anti-scraping). ### AWS WAF Bot Control **Fingerprints**: `aws-waf-token` cookie, `/challenge.js` telemetry endpoint, AWS-branded CAPTCHA page. AWS WAF Bot Control is a managed rule group within the broader AWS WAF product. Its difficulty varies a lot by deployment. Some sites turn on only basic IP rate limits; others turn on full behavior fingerprinting. The `aws-waf-token` must accompany all requests after you solve a challenge. Detection signals: - Bot Control managed rules use ML classification; deployment difficulty depends on which rules the site operator enabled - You must send `aws-waf-token` on all subsequent requests; without it, the site blocks each request - AWS WAF is a full Layer 7 WAF; bot control is one module alongside DDoS protection, rate limiting, and custom rules python ``` result = client.scrape(ScrapeConfig(url="https://web-scraping.dev/antibot/easy", asp=True)) # 96% success rate ``` No dedicated AWS WAF bypass guide exists yet. See the [Scrapfly AWS WAF bypass page](https://scrapfly.io/bypass/aws-waf) for current approach notes. ## Cross-Vendor Comparison: Choosing Your Strategy ### Difficulty and Maintenance Burden Matrix Vendor Detection Sophistication DIY Bypass Difficulty Scrapfly Success Rate DIY Maintenance Burden Cloudflare High Hard 98% Medium-High Akamai High Challenging 97% High DataDome Moderate Hard 96% Medium Incapsula Moderate Medium-Hard 96% Medium AWS WAF Variable Low-High 96% Low-High PerimeterX High Challenging 95% High F5 BIG-IP Moderate Hard 95% Medium Kasada Extreme Extreme ~94% High AWS WAF's wide difficulty range reflects how much the deployed rule set varies by customer. A site using only rate limiting is much easier than one using full behavior analysis. Kasada is the only vendor with Extreme ratings for both detection sophistication and DIY bypass difficulty. It's also the only one where DIY isn't viable in production. ### DIY vs. Scraping API: When Each Makes Sense DIY works when you're targeting one vendor, at low volume, with engineering time available to maintain the bypass. It's a reasonable choice for internal tooling or personal projects where an occasional failure is acceptable. A scraping API makes sense when you're hitting multiple vendors, running at production scale, or working with a small team. The maintenance burden is the hidden cost DIY calculations usually ignore. Bypass maintenance is ongoing, not one-time, and each vendor update can break your approach. If you are choosing the access layer rather than the vendor, use our [proxy-versus-web-unblocker decision guide](https://scrapfly.io/blog/posts/how-to-choose-the-best-proxy-unblocker). ### Cost Comparison: DIY vs. Scrapfly Approach 25K req/mo 100K req/mo 1M req/mo DIY headless + datacenter proxies Low Low–Medium Medium–High DIY headless + residential proxies Medium Medium–High Extreme Scrapfly ASP Low Medium High Engineering maintenance (all DIY) 4–8 hrs/mo 4–8 hrs/mo 8–16 hrs/mo Datacenter proxy costs look low until you factor in success rates. Against Kasada or Akamai, datacenter proxies succeed below 20% of the time. The effective cost per successful request climbs far above what the tier suggests. For Scrapfly ASP pricing at each volume, see the [pricing page](https://scrapfly.io/pricing). ## Using Scrapfly Across All Eight Anti-Bot Systems Scrapfly's Anti-Scraping Protection (ASP) handles all eight anti-bot vendors with one parameter. You don't need separate implementations per vendor. The same `asp=True` flag routes your request through the right bypass stack automatically. ASP manages these signals for you across all eight vendors: - TLS/JA3 and HTTP/2 fingerprinting: your requests look like real browser connections - Residential proxy rotation across 190+ countries: no requests go out from flagged ASN ranges - Browser fingerprinting: canvas, WebGL, hardware sensors, and behavior signals all match - CAPTCHA solving: ASP solves challenges automatically during the bypass flow - Session management: ASP tracks cookies and clearance tokens across requests The code pattern is the same regardless of vendor: python ``` from scrapfly import ScrapflyClient, ScrapeConfig client = ScrapflyClient(key="YOUR_SCRAPFLY_KEY") # Same asp=True pattern regardless of the target's anti-bot vendor. # Swap these demo URLs for the protected sites you're authorized to scrape. for url in [ "https://web-scraping.dev/antibot/easy", # anti-bot challenge (403 without asp) "https://web-scraping.dev/blocked", # block page served with a 200 status "https://web-scraping.dev/product/1", # standard product page "https://web-scraping.dev/products", # product listing ]: result = client.scrape(ScrapeConfig( url=url, asp=True, country="US", render_js=True, )) print(result.scrape_result["content"][:200]) ``` Success rates by vendor: Vendor Success Rate Cloudflare 98% Akamai 97% DataDome 96% Incapsula 96% AWS WAF 96% PerimeterX 95% F5 95% Kasada ~94% The spread is narrow, and that's the point. Cloudflare and Akamai sit highest because their challenges are widely mapped and stable. Kasada stays a few points lower; its rotating interrogation model is the hardest to keep pace with. ASP also retries on failure, so the effective rate across a few attempts runs higher than any single number above. Vendor-specific bypass pages with current approach notes: - [Cloudflare bypass](https://scrapfly.io/bypass/cloudflare) - [Akamai bypass](https://scrapfly.io/bypass/akamai) - [DataDome bypass](https://scrapfly.io/bypass/datadome) - [Incapsula bypass](https://scrapfly.io/bypass/incapsula) - [AWS WAF bypass](https://scrapfly.io/bypass/aws-waf) - [PerimeterX bypass](https://scrapfly.io/bypass/perimeterx) - [F5 bypass](https://scrapfly.io/bypass/f5) - [Kasada bypass](https://scrapfly.io/bypass/kasada) For full ASP documentation: [Scrapfly Anti-Scraping Protection docs](https://scrapfly.io/docs/scrape-api/anti-scraping-protection). ### Web Scraping API Scrape any website with our powerful API. Anti-bot bypass, JavaScript rendering, and rotating proxies built-in. ## Summary Anti-bot bypass starts with identification. Once you know which vendor you're dealing with from headers, cookies, and JS signatures, you can pick the right approach for that detection model. Kasada is the only vendor where DIY isn't viable in production. For the others, DIY is possible at small scale with the right proxy setup and browser fingerprinting. But across all eight vendors, maintenance is ongoing. Each vendor update can break an existing bypass, and keeping a multi-vendor DIY stack working is a continuous engineering investment. Scrapfly's ASP covers all eight vendors with one parameter. If you're hitting multiple vendors or scraping at production scale, the cost comparison in this guide usually makes the decision clear.