Audit & provenance

For every result, kwery can hand you the complete trail of where the data came from: which URLs were fetched, when, from which country, how long each fetch took, and what HTTP status the marketplace returned. This is the audit backbone for MAP enforcement, compliance work, and any situation where "we saw this price" needs to be provable, not just stated.

Rolling out. The include_meta provenance block is being enabled account by account and may not yet be attached to your results. Treat meta as optional in your pipeline for now, and contact us if your workflows need the crawl trail today — we'll prioritize your account. The shape documented below is final.

• "Prove this price existed." — the exact source URL plus an observation timestamp, per fetch.
• "When exactly was this observed?" — observed_at, ISO 8601, UTC, per fetch.
• "Was this seen from the right market?" — geo, the country the request was made from (a .de price observed from de, not through some other locale).
• "Is this data point disputable?" — every fetch has a stable crawl_id; quote it to support and we can trace that individual fetch end to end.

Provenance is opt-in per request, so you only carry the extra payload when you need it:

• Batch (POST /job) — set include_meta: true on the request.
• Stream (POST /stream) — set include_meta: true; the provenance block is attached to each result in every webhook delivery.

curl -s https://api.kwery.co/job \
  -H "Authorization: Bearer $KWERY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "source": "idealo",
    "country": "de",
    "key": "gtin",
    "values": ["4006381333962"],
    "include_meta": true
  }'

There is no extra charge — provenance rides along with the result you already paid for.

With include_meta enabled, each result carries meta.crawls[] — one entry per fetch that contributed to the result:

"meta": {
  "crawls": [
    {
      "crawl_id": "9f1c2d3e4b5a6c7d8e9f0a1b",
      "url": "https://www.idealo.de/preisvergleich/OffersOfProduct/...",
      "http_status": 200,
      "duration_ms": 2341,
      "observed_at": "2026-07-01T12:00:00Z",
      "geo": "de",
      "job_type": "idealo.search"
    }
  ]
}

Most marketplaces need more than one fetch to assemble a complete answer, and every fetch appears as its own entry:

• A GTIN lookup typically runs a search fetch (resolve the GTIN to the marketplace's product) plus a product/offer fetch.
• search_and_offers adds a seller-list fetch on sources where offers live on a separate endpoint (Amazon, Kaufland).
• Paginated offer lists (Idealo) add one entry per extra page fetched.
• A cache-served result carries the trail of the original crawls — observed_at shows you exactly how old the observation is (the same age the trust freshness component scores).

Read together, the entries are the full story of how your answer was produced.

You don't need include_meta to know whether a result was fetched for you or re-served from cache — every result carries two fields regardless:

With include_meta, observed_at equals the observed_at of the last entry in meta.crawls[].

• MAP / pricing disputes — store url + observed_at + crawl_id alongside the price. A violation report that says "€189.00 on idealo.de, observed 2026-07-01 12:00 UTC from Germany, crawl 9f1c…" is evidence; a bare price is an anecdote.
• Compliance & retention — results (with their provenance) are downloadable for 72 hours; persist them on your side as part of your evidence chain. Enterprise plans add managed audit-trail export — the complete provenance stream delivered to your storage — see Pricing → Enterprise.
• Debugging with support — anything odd about a data point, send us the crawl_id. It pins the conversation to one specific fetch instead of "sometime yesterday".

Confidence score and provenance answer different questions about the same result: trust scores how much to rely on it; provenance proves where it came from. High-stakes pipelines typically use both: route on trust.score, archive meta.crawls for whatever you act on.

• Does it cost extra? No — no extra credits. It's off by default only to keep payloads small.
• How much bigger are payloads? One crawl entry is ~200 bytes; a typical result carries one to three entries (plus one per extra Idealo offer page).
• Can I get provenance retroactively for a past result? Within the 72-hour retention window, re-download the job — provenance is part of the stored result if the job was submitted with include_meta: true. It cannot be reconstructed for jobs submitted without it, so watch lists that may ever need evidence should always set it.
• Is the raw HTML available? Not via the API. For enterprise evidence-chain requirements (stored page snapshots), talk to us.

• Confidence score — the confidence score carried by every result.
• Change events — notifications when watched products change.
• Webhooks — stream deliveries, where provenance is attached per result.