Anti-bot protection

Many sites sit behind an anti-bot service (Cloudflare, DataDome, Akamai, PerimeterX, Imperva) that refuses anything that doesn't look like a person in a browser. The API handles them in three layers, all on by default.

1. Look like a browserCopy Anchor Link

Every request without a browser carries a real browser's fingerprint: User-Agent, headers, TLS and HTTP/2 settings. That alone gets through many walls: a page like https://www.leboncoin.fr/c/ameublement, protected by DataDome, loads without anything else.

2. Recognize the wallCopy Anchor Link

When a site refuses the page, the API tells which service did it, from its headers, cookies and challenge page:

{
  "error": {
    "code": "HTTP_ERROR",
    "message": "HTTP 403 for https://example.com/ (blocked by datadome)",
    "status": 403,
    "blocked_by": "datadome"
  }
}

blocked_by is cloudflare, datadome, akamai, perimeterx, imperva, or unknown for a plain 403 or 429 without a known signature.

3. EscalateCopy Anchor Link

When every strategy of a request is stopped by a wall, the API retries with stronger means, in order:

1. a browser: Chromium with the stealth plugin, which runs the wall's JavaScript challenge;
2. Camoufox, a Firefox built to avoid detection.

Each step keeps the rest of the request (proxy, headers, actions, fields), and a warning says what happened:

"warnings": ["blocked by datadome (HTTP 403): retried with chromium"],
"meta": { "strategy": 1, "browser": { "engine": "chromium", "driver": "playwright", "stealth": "plugin" }, … }

Escalation only starts after your own fallback strategies, and only for walls: a 404 or a timeout isn't escalated. Browsers are slower, so a request that escalates takes seconds rather than milliseconds.

"escalate": false turns it off: the request fails with the block instead.

When it still doesn't passCopy Anchor Link

Anti-bots also judge where the request comes from. Datacenter IPs, like most proxy pools, are often blocked whatever the browser: requests to leboncoin.fr through the demo pool's US datacenter proxies are refused even by Camoufox, while the same requests from a home connection pass. What helps, in order:

• no proxy, or a proxy in the site's country (see GET /proxies/countries for what the pool offers);
• residential or mobile proxies, with your own proxy.list;
• a session, so a challenge solved once isn't asked again.

When the wall is a captcha, the solve_captcha action solves it in the page: reCAPTCHA, Cloudflare Turnstile, hCaptcha, image captchas, and with the AI sliders and image grids. "solve_captchas": true does it automatically, challenge pages included.