API 03 · Vercel + optional worker

Web Scraper

Fetches a page (respecting robots.txt and per-domain pacing) or parses supplied HTML, then applies up to 20 extraction rules. Rules support CSS selectors and a documented XPath subset, returning text, HTML or attributes. Adaptive rules store a structural fingerprint; when the selector later stops matching, OzyScrape scores candidates by tag, text, attributes, classes, ancestry, neighbours and sibling position, and returns `recovered` only above a confidence and margin threshold, otherwise `uncertain` with no values. Concurrent crawling is available through /api/v1/crawls.

POST https://dev.itsozy.lol/api/v1/scrape

Vercel + optional worker. Fetching and extraction run on Vercel. `render: true` (JavaScript rendering) requires the optional browser worker.

Authentication

Send Authorization: Bearer ozy_… with an API key that includes the scrape permission (or *). Signed-in dashboard sessions work in the playground. Each call costs 1 unit.

Parameters

Name	Type		Description
`url`	string	optional	Page to fetch (exactly one of url or html)
`html`	string	optional	Inline HTML up to 2 MiB
`rules[]`	object[]	optional	{name, selector, kind: text\|html\|attribute, attribute?, adaptive?}
`markdown`	boolean	optional	Include a Markdown rendering of the body
`cache_seconds`	integer	optional	Cache the result for up to 3600 s
`render`	boolean	optional	Render JavaScript in the worker before extraction

Example request

curl -X POST https://dev.itsozy.lol/api/v1/scrape \
  -H "Authorization: Bearer $OZY_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "rules": [{"name": "heading", "selector": "h1"}, {"name": "link", "selector": "//a", "kind": "attribute", "attribute": "href", "adaptive": true}], "markdown": true}'

Example response (illustrative)

{
  "cached": false,
  "fields": {
    "heading": {
      "candidate_margin": 1.0,
      "confidence": 1.0,
      "fingerprint": null,
      "match_type": "exact",
      "values": [
        "Example Domain"
      ]
    }
  },
  "markdown": "Example Domain\n==============\n…",
  "metadata": {
    "title": "Example Domain"
  },
  "status": 200,
  "url": "https://example.com/"
}

Related endpoints

Errors

Errors use one envelope: {"error": {"code": "…", "message": "…"}}.

Status	Code	Meaning
403	`destination_blocked`	URL resolves to a private, loopback, metadata or non-standard-port destination
403	`robots_denied`	The site's robots.txt disallows OzyScrape
413	`response_too_large`	Upstream response exceeded 2 MiB
429	`domain_rate_limited`	Wait one second between requests to the same host
502	`upstream_failed`	The site could not be reached
504	`upstream_timeout`	The site did not respond in time
400	`invalid_input`	A parameter is missing or invalid
401	`unauthorized`	No valid API key or session
403	`permission_denied`	The key does not include this API's permission
429	`quota_exceeded`	Free-tier quota reached (20 units/min, 100 units/day)
503	`database_unavailable`	Persistent storage is temporarily unavailable

Rate limits

20 units per minute and 100 units per day per account; outbound fetches to the same host are spaced at least one second apart. Exceeding a limit returns 429.

Try it in the playground