# Submit a stream job

Submit a large batch for incremental webhook delivery. Results are
pushed to `callback_url` in batches as they complete.
## Webhook payload

```json
{
  "job_id": "...",
  "client_ref": "...",
  "batch_sequence": 1,
  "is_final": false,
  "results": [ ... ]
}
```
## Signature verification
Each delivery is signed with HMAC-SHA256. Verify using the
`webhook_secret` returned at submission:

```
X-Signature: sha256=<hex digest>
```
Compute `HMAC-SHA256(webhook_secret, raw_request_body)` and compare.
## Fallback
If webhook delivery fails after all retries, use `POST /stream/{id}/replay-all`
to re-queue dead-lettered deliveries, or retrieve results directly via
`GET /stream/{id}/results`.
## Limits
| Parameter | Default | Maximum |
|  --- | --- | --- |
| `values` | — | 100 000 |
| `delivery_batch_size` | 100 | 200 |
| Job timeout | 1 440 min | 1 440 min |

Endpoint: POST /stream
Version: 4.0
Security: ApiKeyAuth, BearerAuth, BasicAuth

## Description:

  - `Webhook payload` ()
    {
  "job_id": "...",
  "client_ref": "...",
  "batch_sequence": 1,
  "is_final": false,
  "results": [ ... ]
}
{
  "job_id": "...",
  "client_ref": "...",
  "batch_sequence": 1,
  "is_final": false,
  "results": [ ... ]
}

  - `Signature verification` ()
    Each delivery is signed with HMAC-SHA256. Verify using thewebhook_secret returned at submission: X-Signature: sha256=<hex digest>
X-Signature: sha256=<hex digest> Compute HMAC-SHA256(webhook_secret, raw_request_body) and compare.

  - `Fallback` ()
    If webhook delivery fails after all retries, use POST /stream/{id}/replay-allto re-queue dead-lettered deliveries, or retrieve results directly viaGET /stream/{id}/results.

  - `Limits` ()
    ParameterDefaultMaximumvalues—100 000delivery_batch_size100200Job timeout1 440 min1 440 min

## Security:

  - `ApiKeyAuth` (unknown)
    apiKey in header x-api-key

  - `BearerAuth` (unknown)
    http bearer JWT

  - `BasicAuth` (unknown)
    http basic

## Request fields (application/json):

  - `source` (string, required)
    Enum: "idealo", "amazon", "google", "ebay", "kaufland", "otto", "billiger", "guenstiger", "walmart", "lidl", "aliexpress", "rakuten"

  - `country` (string, required)
    ISO 3166-1 alpha-2 country code (lowercase).
    Example: de

  - `key` (string)
    Type of values being submitted. Valid values depend on `source`.

  - `topic` (string)
    Processing mode. Valid values depend on `source`.

  - `values` (array, required)
    Up to 100 000 values.

  - `max_pages` (integer)
    Number of result pages to fetch per value. Applicable to Idealo and Amazon.
Ignored by Google and eBay.

  - `max_age` (integer)
    Maximum acceptable age of a result, **in seconds**.
Default: `1200` (20 minutes). Maximum: `259200` (3 days) — the
retention period for results; larger values are clamped.
When a value was already crawled inside this window, the stored
result is returned without re-crawling it and the response is not
charged. Set to `0` to always force a fresh crawl.
Available on eligible accounts; elsewhere every value is always
crawled fresh.

  - `sort_by` (string)
    Sort order for offers within each result.

  - `timeout` (integer)
    Job hard timeout in minutes. The job is forcibly finished with
whatever partial results exist after this time. Default: 1 440 (24 h).

  - `callback_url` (string, required)
    Endpoint to receive incremental result batches.

  - `not_found_max_age` (integer)
    How many days a not-found cache entry is honoured before the product
is re-queried. Applies to GTIN and PZN lookups (Idealo `gtin`/`pzn`,
Amazon `gtin`, Google `product`). Set to `1` to force a fresh lookup
on the next job after a single day. Ignored for `term` and `id` keys.

  - `include_meta` (boolean)
    When `true`, each result includes a `meta` provenance block
describing how the data was gathered — the crawl chain with
URL, HTTP status, timing, and observation timestamps.

  - `wait` (any)
    Hold the request open for up to this many seconds (`true` = the
maximum, 120) and, if the job finishes in time, return the results
inline — the response gains a `results` array identical to what
`GET /job/{id}/download` serves. If the job is still running when
the wait elapses, the normal job object is returned with
`wait_timed_out: true`; keep waiting with `GET /job/{id}?wait=N`
or poll `GET /job/{id}` as usual. Accepts exactly one value
(more returns `"error": true`); leave it off for batches.

  - `client_ref` (string)
    Opaque reference string echoed in every webhook payload and in
the status response. Use to correlate jobs with your own records.
    Example: batch-2026-04-03

  - `delivery_batch_size` (integer)
    Number of results per webhook delivery.

  - `fetch_seller_names` (boolean)
    Idealo only. When `true`, seller detail pages are crawled for
marketplace offer sellers whose names are not yet in the database.
Seller names already cached in the database are always applied
regardless of this flag.
Enabling this adds one additional request per unresolved seller,
billed at the standard rate.

## Response 200:

  - `200` (unknown)
    Stream job created

## Response 200 fields (application/json):

  - `error` (boolean)
    Example: false

  - `job_id` (string)

  - `webhook_secret` (string)
    32-byte hex secret for verifying webhook signatures.
Store securely — not retrievable after submission.

  - `client_ref` (string)

  - `message` (string)
    Example: country not supported by source

## Response 401:

  - `401` (unknown)
    Missing or invalid credentials.

## Response 401 fields (application/json):

  - `error` (string)
    Example: user not found

  - `message` (string)

## Response 403:

  - `403` (unknown)
    Authenticated but not subscribed to this source/country.

## Response 403 fields (application/json):

  - `error` (string)
    Example: user not found

  - `message` (string)