Scrape with AI
Instead of inspecting HTML and writing CSS selectors, you write a sentence — "the product name, price and whether it's in stock" — and a language model reads the rendered page to fill it in. It exists for the pages that break every selector-based scraper: inconsistent markup, A/B-tested layouts, or sites you've never seen before.
Run it online
Run this on our servers with your account. Free tools run in your browser; this one bills your KIT balance per the price above.
The problem: layouts move, selectors break
Anyone who has run a selector-based scraper for more than a few months knows the routine — a site redesign ships, a class name changes, and a monitoring job silently starts returning nulls. This endpoint targets exactly that failure mode, along with the one-off case where you need data from a page today and don't want to spend twenty minutes in dev tools finding the right nesting to target.
How the extraction actually works
You call POST /web/scrape-ai with a URL and a plain-language description of the fields you want; the task_id comes back instantly while, behind it, the page is fetched, its rendered content is prepared for a language model, and that model is instructed to return only the fields you described, in JSON, grounded in what's actually present on the page rather than invented from general knowledge.
Why natural-language extraction is now practical
Large language models only became reliable enough to follow structured-output instructions over ordinary text in the last few years; before that, the only options were brittle selectors or hand-written regular expressions for every new domain. Describing intent in a sentence — rather than encoding it as a CSS path — means the same request can survive a site redesign that would have silently broken a selector map.
What to expect in the result and when things don't match
The response is a JSON object keyed by the field names implied in your description; if the page genuinely lacks a field you asked for, it comes back null rather than a guessed value, because the model is grounded to the page content and instructed not to fabricate. As with every task on the platform, a failed extraction retries automatically up to three times before returning a clear error, and you're only billed once a task completes successfully.
Where this fits alongside the other scraping endpoints
Teams typically reach for this endpoint first when prototyping a new data source, then migrate high-volume, stable pages to selector-based scraping once the layout is known and won't be worth re-describing every run. For cases where you need a strict, machine-checked shape — required fields, fixed types, enumerated values — the schema-scraping endpoint layers that guarantee on top of the same natural-language flexibility.
What you can do with it
Prototyping a new scraping target
Point it at a site you've never scraped before and describe the fields you need, skipping the dev-tools work of mapping selectors before you even know if the source is worth it.
Sites with inconsistent or A/B-tested markup
Extract the same fields reliably from listing pages where different products render with slightly different HTML structures.
Long-tail research across many different domains
Pull comparable fields — author, publish date, headline — from dozens of unrelated news sites without writing a selector map for each one.
Rescuing a broken selector job
Swap a monitoring pipeline over to the AI endpoint after a site redesign breaks its CSS selectors, buying time to decide whether to rebuild the selector map.
FAQ
How is AI web scraping different from selector-based scraping?
You describe the fields you want in plain language instead of writing CSS selectors, which makes it more resilient to layout changes at a somewhat higher per-URL cost.
What does web.scrape_ai cost?
$0.046 per request plus $0.0145 per URL, charged only for tasks that complete successfully.
Can it scrape any website?
It works on any publicly reachable page whose content renders in the response, including JavaScript-heavy pages, though sites that require login are out of scope.
Will it invent data that isn't on the page?
No — the model is grounded to the page's actual content and returns null for fields it genuinely can't find rather than guessing.
Is there a free trial for the AI scraping API?
There's no free tier — free tiers get abused and slow everyone down. Access runs on a prepaid ForHosting KIT balance: top up from $10.00 (it never expires) and each request is charged at its published price, so a call with no balance returns HTTP 402. No subscription, no tokens, no invented credits, and a failed task is never charged.
How do I write a good extraction description?
Name each field plainly and note its expected form when it matters, for example "price as a number without currency symbol" — specificity in your description improves match quality.
How do I receive the results?
Via a signed webhook when the task completes, or a signed link that stays valid for 24 hours if you'd rather fetch it yourself.
When should I use scrape_schema instead of scrape_ai?
Reach for scrape_schema when you need a strict, validated data shape with required fields and fixed types; use scrape_ai when a flexible plain-language description is enough.
For developers — API access
Everything on this page is available programmatically. This section is for teams who want to wire it into their own systems; everyone else can just use the tool above.
API endpoint
Prefer to automate it? One authenticated POST creates the task; the result comes back by webhook or a signed link. The same capability also runs here on the web, and soon from our app, email and Telegram.
Call it from your stack
curl -X POST https://api.kit.forhosting.com/web/scrape-ai \
-H "Authorization: Bearer $KIT_KEY" \
-H "Content-Type: application/json" \
-d '{"url":"https://ejemplo.com"}'const res = await fetch("https://api.kit.forhosting.com/web/scrape-ai", {
method: "POST",
headers: {
"Authorization": `Bearer ${process.env.KIT_KEY}`,
"Content-Type": "application/json"
},
body: JSON.stringify({
"url": "https://ejemplo.com"
})
});
const { task_id } = await res.json();import os, requests
res = requests.post(
"https://api.kit.forhosting.com/web/scrape-ai",
headers={"Authorization": f"Bearer {os.environ['KIT_KEY']}"},
json={
"url": "https://ejemplo.com"
},
)
task_id = res.json()["task_id"]<?php
$res = file_get_contents("https://api.kit.forhosting.com/web/scrape-ai", false, stream_context_create([
"http" => [
"method" => "POST",
"header" => "Authorization: Bearer " . getenv("KIT_KEY") . "\r\nContent-Type: application/json",
"content" => '{"url":"https://ejemplo.com"}',
],
]));
$task = json_decode($res, true);body := bytes.NewBufferString(`{"url":"https://ejemplo.com"}`)
req, _ := http.NewRequest("POST", "https://api.kit.forhosting.com/web/scrape-ai", body)
req.Header.Set("Authorization", "Bearer "+os.Getenv("KIT_KEY"))
req.Header.Set("Content-Type", "application/json")
res, _ := http.DefaultClient.Do(req)Example request
{
"url": "https://ejemplo.com"
}Example response
{
"task_id": "tsk_a1b2c3d4e5f6a1b2c3d4e5f6",
"type": "web.scrape_ai",
"status": "queued",
"_links": {
"result": "/tasks/tsk_…/result"
}
}The API is asynchronous: the call returns a task_id immediately and the result arrives by webhook. Polling is capped at 1 req/s per task.
Pricing
Published price — no tokens, no invented credits. A failed task is never charged.
Limits
timeout_sec | 30 |
max_crawl_pages | 25 |
Errors
| HTTP | Code | Meaning |
|---|---|---|
401 | unauthorized | Missing or invalid API key. |
402 | insufficient_balance | Your balance doesn't cover the task price. |
404 | unknown_type | That task type doesn't exist. |
429 | rate_limited | Too many requests. Use the webhook instead of polling. |
422 | task_failed | The task failed after 3 retries. You are never charged for it. |