---
source_url: "https://scrapfly.io/ai-web-scraping-api"
title: "Scrapfly is the Best AI Web Scraping API: Scrape and Extract in One Call"
mirrored_at: 2026-08-13T03:35:36.521Z
host: scrapfly.io
cited_in_42a: true
mirror_canonical: "https://index.42a.ai/scrapfly.io/ai-web-scraping-api"
---

> **Original source:** https://scrapfly.io/ai-web-scraping-api

// PRODUCT

Scrapfly is the best AI web scraping API. Scrape and extract structured data in a single API call: anti-bot bypass, JavaScript rendering, and AI-powered extraction - all baked into one endpoint.

## One call. Scraped HTML + structured JSON.

-   **Combined pipeline.** No two-step scrape-then-extract workflow. One request returns both the raw page and AI-extracted data in the same response.
-   **19+ pre-trained models built in.** Pass `extraction_model='product'` and get title, price, images, and more - no selectors to write.

1,000 free credits. No credit card required.

* * *

## 1

API call returns scrape + extracted JSON

## 19+

pre-trained AI extraction models

## 98%

success on Cloudflare-blocked targets

## 55k+

developers building on Scrapfly

* * *

CAPABILITIES

## Scrape and Extract in One Pipeline

Anti-bot bypass, rendering, pre-trained models, LLM prompts, and template rules - all composable, all on one endpoint.

### From URL to Structured JSON - One Request

The traditional workflow requires two calls: one to fetch the page, one to parse it. This API collapses both into a single request. Anti-bot bypass, JavaScript rendering, and AI extraction run in sequence on the same endpoint. The response carries both the raw scraped content and the structured `extracted_data` object together.

**URL + Schema / Prompt** one call: url, extraction\_model or extraction\_prompt or extraction\_template

**ASP Fetch** anti-bot bypass via Curlium + Scrapium, residential proxies, challenge solver

**Render and Parse** optional JS rendering, screenshot capture, browser\_data XHR capture

**LLM Extraction** vision + DOM + content-aware reasoning over the fetched page

**Schema Validator** output conforms to model schema or your JSON schema; data\_quality coverage report included

**Structured JSON** result.extracted\_data alongside raw HTML, screenshots, log\_url, and cost headers

extracted\_data

data\_quality

[View extraction docs →](https://scrapfly.io/docs/scrape-api/extraction)

### Pre-Trained AI Models

Set `extraction_model` and get a predictable JSON schema for your page type. No selectors to write, no XPath to maintain. The model applies across any domain that matches the schema - same field names, same output shape, every time.

**19+** preset models

**any domain** same schema

**data\_quality** coverage score

[View pre-trained model docs →](https://scrapfly.io/docs/extraction-api/automatic-ai)

### LLM and Schema Extraction

Two LLM-powered modes beyond pre-trained models. Pass a JSON Schema with `extraction_prompt` and the LLM shapes its output to that structure. Or pass a free-form natural-language instruction and get back a JSON object matching your question. Both modes use vision, DOM, and content signals together.

**JSON Schema** structured output

**Freeform** any question

**Vision** \+ DOM aware

[View LLM prompt docs →](https://scrapfly.io/docs/extraction-api/llm-prompt)

### Template Extraction

Define your own CSS, XPath, or JMESPath selectors in a JSON template. Chain type extractors and formatters on any field for precise, repeatable output from a known site structure. Deterministic - same input always yields the same shape.

**CSS** selectors

**XPath** expressions

**JMESPath** JSON queries

type extractors

field formatters

nested fields

repeating rows

[View template docs →](https://scrapfly.io/docs/extraction-api/rules-and-template)

### Anti-Bot and Proxy Built In

Every request inherits the full Web Scraping API bypass stack. Add `asp=true` and Scrapfly detects the active protection layer, assembles a coherent fingerprint, and solves any challenge before extraction runs. Failed challenge retries do not cost credits.

**Curlium** HTTP fingerprint

**Scrapium** browser identity

**190+ countries** proxy coverage

**Free retries** on challenge fail

[View ASP docs →](https://scrapfly.io/docs/scrape-api/anti-scraping-protection)

### Extraction Result Cache

Extraction results are cached per (URL, schema) pair. Identical requests return the cached structured JSON without re-running the LLM. The raw scrape still runs fresh; only the extraction layer is served from cache when the input is unchanged.

**Per (URL + schema)** cache key

Faster repeat calls

Lower extraction cost on reruns

### Observability Built In

Every response carries a `log_url`. Follow it to inspect the full request, response headers, rendered HTML, HAR waterfall, and screenshots. The `content_replay_url` lets you re-run extraction against the stored HTML without making another scrape call. The `data_quality` field on `extracted_data` reports field-level coverage so low-confidence extractions are visible before they reach your pipeline.

**log\_url** full trace

**Replay** no extra scrape

**data\_quality** field coverage

**Cost headers** per request

log\_url

content\_replay\_url

data\_quality

X-Scrapfly-Api-Cost

[View debug and observability docs →](https://scrapfly.io/docs/scrape-api/debug)

### Works With Your AI Stack

The API returns plain JSON, which plugs into any framework that can consume a URL. LangChain and LlamaIndex wrappers are available in the official SDKs. For RAG pipelines, the structured response is already chunked by field - no further parsing required. Custom agent stacks call the endpoint directly with the same `api_key` used for all other Scrapfly products.

**LangChain** SDK integration

**LlamaIndex** SDK integration

**RAG pipelines** pre-structured

**Agent stacks** plain HTTP

Python SDK

TypeScript SDK

HTTP / cURL

Research assistants

Data pipelines

Training data prep

### Related APIs

The AI Web Scraping API is the combined product. Each layer is also available standalone for tighter control.

### Preset Models at a Glance

Each preset maps a page type to a documented JSON schema with named fields. Pass the model name as `extraction_model` and get back the same structure from any URL matching that type, regardless of domain or layout.

[View all preset model schemas →](https://scrapfly.io/docs/extraction-api/automatic-ai)

### Output Format

The extraction result lands in `result.extracted_data` alongside the scrape. The raw response body is still present - use markdown or screenshot format for LLM context, or HTML for downstream parsing.

JSON (extracted\_data)

Markdown body

HTML body

Screenshot

browser\_data

### Scale With the Crawler API

The single-URL endpoint handles individual requests. When you need the same extraction across an entire site - product catalog, news archive, property listings - pair it with the Crawler API. The crawler handles discovery, deduplication, and scheduling; each page passes through the same extraction pipeline.

[Crawler API →](https://scrapfly.io/products/crawler-api)

### Data Privacy

Document content is processed in memory and discarded after the response is returned. Scrapfly does not store, share, or use your extracted data or page content for training AI models. See the privacy policy for full details.

In-memory processing

Not used for AI training

* * *

CODE

## Scrape + Extract in One Call

Combine anti-bot bypass with AI extraction in a single request. Pick your extraction strategy.

[AI Auto Models](#awsa-strat-auto) [LLM + JSON Schema](#awsa-strat-llm-structured) [LLM Freeform Prompt](#awsa-strat-llm-prompt)

Pre-trained schemas for product, article, review, job\_posting, and more.

```
# pip install scrapfly-sdk[all]

from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse

client = ScrapflyClient(key="API KEY")

api_response: ScrapeApiResponse = client.scrape(
    ScrapeConfig(
        url='https://web-scraping.dev/product/1',
        # what object to scrape? product, review, real estate listing etc.
        extraction_model="product",
    )
)
print(api_response.scrape_result['extracted_data']['data'])
```

```
import { 
    ScrapflyClient, ScrapeConfig 
} from 'jsr:@scrapfly/scrapfly-sdk';

const client = new ScrapflyClient({ key: "API KEY" });
let api_result = await client.scrape(
    new ScrapeConfig({
        url: 'https://web-scraping.dev/product/1',
        // what object to scrape? product, review, real estate listing etc.
        extraction_model: "product",
    })
);
console.log(api_result.result.extracted_data);
```

```
http https://api.scrapfly.io/scrape \
key==$SCRAPFLY_KEY \
url==https://web-scraping.dev/product/1 \
extraction_model==product
```

Give the LLM a schema, get that shape back. Guaranteed.

```
from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse

client = ScrapflyClient(key="API KEY")

api_response: ScrapeApiResponse = client.scrape(
    ScrapeConfig(
        url='https://web-scraping.dev/product/1',
        # Prompt for specific structured data formats:
        extraction_prompt="Extract product features in JSON format",
    )
)
print(api_response.scrape_result['extracted_data'])
```

```
import { 
    ScrapflyClient, ScrapeConfig 
} from 'jsr:@scrapfly/scrapfly-sdk';

const client = new ScrapflyClient({ key: "API KEY" });
let api_result = await client.scrape(
    new ScrapeConfig({
        url: 'https://web-scraping.dev/product/1',
        // Prompt for specific structured data formats:
        extraction_prompt: "Extract product features in JSON format",
    })
);
console.log(api_result.result.extracted_data);
```

```
http https://api.scrapfly.io/scrape \
key==$SCRAPFLY_KEY \
url==https://web-scraping.dev/product/1 \
"extraction_prompt=Extract product features in JSON format"
```

Natural-language extraction instructions. Use for any layout.

```
from scrapfly import ScrapeConfig, ScrapflyClient, ScrapeApiResponse

client = ScrapflyClient(key="API KEY")

api_response: ScrapeApiResponse = client.scrape(
    ScrapeConfig(
        url='https://web-scraping.dev/product/1',
        # Use any LLM prompt:
        extraction_prompt="What's price of the product?",
    )
)
print(api_response.scrape_result['extracted_data'])
```

```
import { 
    ScrapflyClient, ScrapeConfig 
} from 'jsr:@scrapfly/scrapfly-sdk';

const client = new ScrapflyClient({ key: "API KEY" });
let api_result = await client.scrape(
    new ScrapeConfig({
        url: 'https://web-scraping.dev/product/1',
        // Use any LLM prompt:
        extraction_prompt: "What's price of the product?",
    })
);
console.log(api_result.result.extracted_data);
```

```
http https://api.scrapfly.io/scrape \
key==$SCRAPFLY_KEY \
url==https://web-scraping.dev/product/1 \
"extraction_prompt=What's price of the product?"
```

* * *

LEARN

## Docs, Tools, And Ready-Made Scrapers

Everything you need to go from a URL to a production data pipeline.

### API Reference

Every extraction parameter, every response field, with runnable examples for all three strategies.

[Developer Docs →](https://scrapfly.io/docs/scrape-api/extraction)

### Academy

Interactive courses on web scraping, anti-bot bypass, HTML parsing, and structured data extraction.

[Start learning →](https://scrapfly.io/academy)

### Open-Source Scrapers

40+ production-ready scrapers on GitHub. Each one uses the AI extraction pipeline for the parsing step.

[Explore repo →](https://github.com/scrapfly/scrapfly-scrapers)

### Developer Tools

CSS selector tester, cURL-to-Python, JA3 checker, HTTP/2 fingerprint, and more.

[Browse tools →](https://scrapfly.io/web-scraping-tools)

* * *

* * *

FAQ

## Frequently Asked Questions

### What is the AI Web Scraping API?

The AI Web Scraping API combines the Web Scraping API and the Extraction API into a single call. You send a URL and get back both the scraped page content and AI-extracted structured JSON in the same response. Anti-bot bypass, JavaScript rendering, residential proxies, and all three extraction strategies (pre-trained models, LLM prompts, CSS/XPath templates) are available on the same endpoint.

### How is this different from calling the Web Scraping API and Extraction API separately?

The two-step approach requires two API calls, two sets of credentials to manage, and code to pipe the HTML from one response into the next request. The AI Web Scraping API does both in one call, returning `extracted_data` alongside the raw scrape result. It also inherits the full feature set of the Web Scraping API - asp bypass, render\_js, sessions, webhooks - without any additional configuration.

### Which extraction strategy should I use?

Use `extraction_model` when your page matches a standard schema - product pages, news articles, job listings, reviews. The model returns a predictable JSON shape across any domain. Use `extraction_template` when you need precise, repeatable extraction from a known site with a CSS or XPath template you define. Use `extraction_prompt` for ad-hoc questions, rapidly changing layouts, or anything that doesn't fit a fixed schema.

### How many pre-trained models are available?

There are 19+ pre-trained models covering the most common page types: product, article, review, job posting, real estate listing, recipe, event, and more. Every model returns a documented JSON schema with a `data_quality` coverage report so you can detect low-confidence extractions before they reach your pipeline.

### Does anti-bot bypass work with AI extraction?

Yes. Add `asp=true` to any call and the API handles Cloudflare, DataDome, Akamai, PerimeterX, Kasada, Imperva, F5, and AWS WAF - the same bypass stack as the Web Scraping API. Extraction runs on the successfully retrieved page content, so blocked sites work exactly the same as unblocked ones.

### Is my data used for AI training?

No. Scrapfly does not store, share, or use your document content for training AI models. Data is processed in memory and discarded after the response is returned. See the privacy policy for full details.

### Is web scraping legal?

Scraping publicly accessible data is legal in most jurisdictions (Meta v. Bright Data and hiQ v. LinkedIn have established strong precedent). You are responsible for respecting robots.txt, rate limits, and target terms of service. See [our legal overview](https://scrapfly.io/is-web-scraping-legal) for details.

* * *

**Free tier**

1,000 free credits on signup. No credit card required.

**Pay on success**

You only pay for successful requests. Failed calls are free.

**No lock-in**

Upgrade, downgrade, or cancel anytime. No contract.

### Need more control? We unbundle the stack.

The AI Web Scraping API is the batteries-included product. Each layer is also available standalone: [Web Scraping API](https://scrapfly.io/products/web-scraping-api) for [anti-bot bypass](https://scrapfly.io/bypass) and proxies without extraction, [Extraction API](https://scrapfly.io/products/extraction-api) for structured parsing from HTML you already have, [Browser API](https://scrapfly.io/products/cloud-browser-api) for hosted Playwright / Puppeteer, [AI Browser Agent](https://scrapfly.io/products/ai-browser-agent) for autonomous agent loops, [Scrapium](https://scrapfly.io/scrapium) for stealth Chromium you drive directly, or [Curlium](https://scrapfly.io/curlium) for byte-perfect HTTP with Chrome TLS fingerprints.

[Get Free API Key](https://scrapfly.io/register)

1,000 free credits. No card.