Reliable batch jobs

Live
Control concurrency, retries, reconciliation, and cost across many requests.
Open in ChatGPT
(opens in new tab)
Last verified: 2026-08-11

Queue the work

Create one durable record per input with an internal job ID, model ID, payload hash, state, attempt count, and ModelRush request ID. Workers should claim jobs with bounded concurrency.

Classify retries

Retry 429 and transient 5xx responses with jitter. Move validation and billing failures to a review state. For async media, reconcile an existing generation ID before creating a replacement.

Measure usable output

Track total attempts, successful outputs, accepted outputs, latency, and spend. Cost per usable output is more informative than price per request.