Multi-step requests

One request, several scrapes in a row. Keep sending target_url, fields and options for a single scrape, or add steps to chain several, each with its own configuration:

{
  "target_url": "https://example.com/login",
  "options": { "proxy": { "country": "FR" } },
  "steps": [
    { "fields": ["#welcome"], "options": { "browser": true, "actions": [ … ] } },
    {
      "target_url": "https://example.com/orders",
      "fields": { "orders": { "selector": ".order", "multiple": true } }
    },
    {
      "target_url": "https://example.com/invoices",
      "fields": ["table"],
      "options": { "pdf": true }
    }
  ]
}

Steps run in order and share one session: cookies set by a step are sent by the next ones, whether they use a browser or not.

Log in, then scrapeCopy Anchor Link

Step 1 logs in with a browser, which the demo's login form needs. Step 2 reads the account page with a plain, fast request, logged in by the cookie step 1 got:

Several pages, one configurationCopy Anchor Link

Set what the steps share at the top level, and only what differs in each step:

To use what a step extracted in a later step (a link, a token, an ID), see Variables.

How a step is builtCopy Anchor Link

Every option works in a step, fallback and cache_ttl included. A top-level session applies to every step; "session": true creates one name for all of them.

Continue on the same pageCopy Anchor Link

A step with "continue": true doesn't load anything: it carries on from where the previous step left the page. In a browser, it's the same tab, with what the previous actions changed (a form filled, a captcha solved, a list expanded). Without a browser, its fields are read from the previous step's HTML, for free.

• A step that continues can't have a target_url, and the first step can't continue.
• It's tried once: no retries, fallbacks or escalation (they'd load the page again).
• Switching from one browser engine to another loads the previous step's final URL in the new one, with a warning; a step without a browser can't run actions on the previous page.
• If the previous step failed, it fails with NOTHING_TO_CONTINUE.

The responseCopy Anchor Link

steps holds one result per step, each shaped like a single scrape's response:

{
  "steps": [
    {
      "target_url": "…/demo/login",
      "final_url": "…/demo/account",
      "status": 200,
      "browser": { "engine": "chromium", … },
      "data": ["Logged in as demo"],
      …
    },
    {
      "target_url": "…/demo/account",
      "status": 200,
      "browser": null,
      "data": { "orders": ["#1001 Desk lamp", "#1002 Mug"] },
      …
    }
  ]
}

When a step failsCopy Anchor Link

By default, the first failing step stops the request. The response has its HTTP status and error, after the results of the steps before it:

{
  "steps": [
    { "target_url": "…", "status": 200, "data": [ … ] },
    {
      "target_url": "…",
      "error": { "code": "HTTP_ERROR", "message": "HTTP 404 for …", "status": 404 }
    }
  ]
}

With "continue_on_error": true at the top level, every step runs, failed steps carry their error, and the response is 200. On a step, it applies to that step only: its failure doesn't stop the next ones (a step that continues a failed one still fails, with NOTHING_TO_CONTINUE).

"steps": [
  { "fields": ["h1"], "options": { "actions": [{ "click": "#cookie-ok" }] }, "continue_on_error": true },
  { "target_url": "https://example.com/list", "fields": { "items": { "selector": ".item", "multiple": true } } }
]

A step doesn't need fields when it does something else: actions, a screenshot, a PDF or AI extraction.

Up to 10 steps per request. They don't share a time limit: each has its own timeout per attempt.