PRACTICAL GUIDES / UPDATED 2026-11-15

Operate a CPU Exporter and Verifier

Start with one bounded export queue, a resident export worker and a separate public verifier process. This is a deployment recipe around the existing SDK, not a built-in durable queue or a claim of horizontal scaling.

A small startup pipeline

generation → final rendition → bounded job queue → resident SDK exporter
                                             → accepted staging file + receipt
                                             → one publication step

public upload → separate verifier process → reviewed registry / signed report

Keep publisher private keys with the exporter. Give the verifier its own report key and read-only registry access. One SDK shares its Node worker across video profiles and verification; an export can hold that worker lock for the whole job. A public verifier in another process avoids waiting behind your export queue.

Job states and back pressure

State Product behavior
Pending Keep job ID, generation time, input hash, profile and intended rendition in durable application storage
Queue full Decline admission or return a bounded retry response; do not create an unlimited in-memory queue
Running Lease one job to one worker; create a unique staging path for this attempt
Accepted Retain the accepted receipt and reconcile registry/output state before publishing
Published Record publication against the same idempotency key so a retry cannot publish twice
Failed or interrupted Preserve the failure; inspect completed artifacts before retrying; never publish unmarked input as fallback

Set your own maximum queue length, lease timeout and retry cap from the workload. A 2-vCPU benchmark is not a measured concurrency limit. The owned-file worker processes a bounded manifest serially; your application supplies durable admission and job state.

Handle a busy verifier

The reference verifier has one inference slot, four upload slots and no queue. Handle HTTP 429 and its Retry-After response with bounded backoff. Do not repeatedly upload a large file in a tight retry loop. Match the application upload limit and your reverse proxy limit; the documented defaults are 512 MiB and 128 MiB respectively.

Size positive, negative and incomplete searches separately. On the ten-second 720p engineering fixture, warm positive verification took 2.04 seconds and a completed negative took 40.32 seconds. Longer searches can exhaust the budget; not_checked is not a completed negative. See the full timing and resource scope.

Keep registry state consistent

Retain the registry, manifests, receipts and application job state on persistent storage. Use a controlled writer and give SQLite readers access to the database directory and its supporting files. Do not copy only a live database file and assume it is a consistent backup.

For a small SQLite deployment, an online backup can use the standard backup API:

from pathlib import Path
import sqlite3

source = Path("registry/assets.sqlite3").resolve()
backup = Path("backups/assets.snapshot.sqlite3")
backup.parent.mkdir(parents=True, exist_ok=True)
# Use a fresh destination name for each snapshot; retain keys separately.
if backup.exists():
    raise FileExistsError(backup)
with sqlite3.connect(source.as_uri() + "?mode=ro", uri=True) as src:
    with sqlite3.connect(backup) as dst:
        src.backup(dst)
        assert dst.execute("PRAGMA integrity_check").fetchone()[0] == "ok"

The database snapshot does not automatically back up referenced manifests or keys. Coordinate their retention, protect private material, and test a restore in isolation. Check that an accepted receipt and media file resolve to the intended public record after restart. Reconcile orphan staging files against your job ledger before applying a retention policy; do not delete unknown files automatically.

A first operational acceptance pass

  1. Submit one image and one video through the same application queue.
  2. Trigger a controlled failed export; confirm no unmarked publication fallback.
  3. Restart the worker and retry the same application job ID; confirm one publication.
  4. Exercise a busy verifier and a clean negative, recording retry behavior and memory.
  5. Restore an isolated registry/manifests snapshot and independently verify an accepted output.

These checks complement the SDK qualification. They do not establish your production traffic capacity or a multi-replica storage design.

First integration · Reference limits · Production setup · Costs

Browse documentation ↗