SE Home
Help
  • Documentation
  • Help centre
SEScrape Engine

Getting started

DocumentationQuickstartAuthenticationPlayground

Extracting data

Selecting valuesTransformsLists & tablesPage dataAI extractionMCP server

Fetching pages

Custom requestsTimeouts & redirectsCachingAutomatic retriesFallback strategies

Browsers & actions

BrowsersActionsRecipesScreenshots & PDFStored files

Getting past blocks

Anti-bot protectionCaptchasProxiesFingerprintsSessions

Workflows

Multi-step requestsVariablesPaginationBatchCrawlingTemplatesSchedules & monitoringAsync jobs & webhooksExport to S3, Sheets, databases & webhooks

Account & billing

CreditsSecretsPlansLimitsLogs & request idsNotificationsAccountAffiliate programmeHelp centre

Reference

POST /scrapeGET /sessions/:idGET /browsersGET /proxies/countriesGET /healthErrors
Documentation

Getting started

DocumentationQuickstartAuthenticationPlayground

Extracting data

Selecting valuesTransformsLists & tablesPage dataAI extractionMCP server

Fetching pages

Custom requestsTimeouts & redirectsCachingAutomatic retriesFallback strategies

Browsers & actions

BrowsersActionsRecipesScreenshots & PDFStored files

Getting past blocks

Anti-bot protectionCaptchasProxiesFingerprintsSessions

Workflows

Multi-step requestsVariablesPaginationBatchCrawlingTemplatesSchedules & monitoringAsync jobs & webhooksExport to S3, Sheets, databases & webhooks

Account & billing

CreditsSecretsPlansLimitsLogs & request idsNotificationsAccountAffiliate programmeHelp centre

Reference

POST /scrapeGET /sessions/:idGET /browsersGET /proxies/countriesGET /healthErrors

Export to S3, Sheets, databases & webhooks

Send every result to your own storage, spreadsheet, database or endpoint, from any request, async job or scheduled scrape.

Add a destination once in the dashboard (Developers → Destinations) or with the API, then add export to any request: once it succeeded, its result is sent there, in the background.

{
  "target_url": "https://example.com/products",
  "fields": { "products": { "selector": ".product", "items": { "name": "h2", "price": ".price" } } },
  "export": "my-bucket"
}
  • "export": "<name>", or { "to": "<name>", "format": "csv" } (S3 only: json, ndjson or csv), or a list of up to 3.
  • It works the same with async jobs and scheduled scrapes: put it in the body you schedule, and every run is exported.
  • The response says what was queued, next to its meta: "export": [{ "destination": "my-bucket", "delivery": 42, "status": "pending" }]. Each destination lists its deliveries, where they went, and why one failed.
  • A delivery that fails is tried again after 1 minute, 5 minutes, 30 minutes and 2 hours. A failed request exports nothing.
  • Each result delivered costs a few credits (see credits); Send a test in the dashboard is free.

Destinations

S3 storage

AWS S3 or any S3-compatible storage (Cloudflare R2, MinIO, Backblaze B2, Wasabi…): a bucket, its region, an access key allowed to PutObject, and for anything but AWS, the endpoint (https://<account>.r2.cloudflarestorage.com). Each result is one object:

<prefix>2026/10/26/<request id>.json

json is the response as is; ndjson one line per page; csv one row per page, or one row per item when a page holds a single list (a listing), with nested values as price.amount columns.

Google Sheets

Rows are appended to a tab, under its header; columns that don't exist yet are added at the end. It uses a Google Cloud service account:

  1. In Google Cloud, create a service account and a JSON key for it, and enable the Google Sheets API.
  2. Share the spreadsheet with the service account's email, as Editor.
  3. Paste the spreadsheet's URL and the JSON key in the destination.

Rows are shaped like the CSV above: when, the page's URL, then one column per value.

Postgres

A connection string (postgres://user:password@host:5432/database) and a table, created when it's missing:

Column
idbigserial
scraped_attimestamptz
request_idthe response's meta.id
target_urltext
statusthe page's HTTP status
datajsonb: what the page extracted
errorjsonb, for a page that failed

TLS is on by default, with the certificate checked. The host must be reachable from the internet: addresses of private networks are refused.

MySQL

The same, for MySQL or MariaDB: mysql://user:password@host:3306/database, the same columns (data and error as json), TLS on by default.

BigQuery

A project, an existing dataset, a table (created when missing, same columns, data as JSON) and a service account key with the BigQuery Data Editor role on the dataset. Rows are streamed, one per page.

Webhook

Any public HTTPS endpoint. Each result is POSTed as JSON:

{ "request_id": "…", "delivered_at": "2026-10-26T09:00:00.000Z", "response": { … } }

signed like async jobs (x-signature: sha256=<HMAC of the body with your webhook secret>), with the headers you add (an authorization token, for instance, kept encrypted). Any 2xx counts as delivered.

With the API

/v1/destinations takes and returns the same as the dashboard, with your API key:

GET /v1/destinationsthe list (credentials never shown)
POST /v1/destinations{ "name", "type": "s3" | "sheets" | "postgres" | "mysql" | "bigquery" | "webhook", … }: bucket, region, endpoint, prefix, format, access_key_id, secret_access_key · spreadsheet, sheet, service_account · url, table, ssl (postgres, mysql) · project, dataset, table, service_account · url, headers (webhook)
PATCH /v1/destinations/{name}change settings; credentials left out are kept
DELETE /v1/destinations/{name}
POST /v1/destinations/{name}/testsends a sample result now: { "ok": true, "location": "…" } or the error
GET /v1/destinations/{name}/deliveriesthe last 50 deliveries

On this page

  • Destinations
  • S3 storage
  • Google Sheets
  • Postgres
  • MySQL
  • BigQuery
  • Webhook
  • With the API