Home/ Guides/ Clef-omni Tutorial: Multimodal Classification with Workers AI
Guides

Clef-omni Tutorial: Multimodal Classification with Workers AI

Build a documentation-based Clef-omni classifier with Python, understand multimodal inputs and pricing, and plan a fair accuracy and latency evaluation.

Published Oct 19, 2026 · 3 min read
Image, audio, text and video inputs passing through a prism into sorted categories

Checked October 18, 2026. Cloudflare announced Clef-omni on October 9. It evaluates multimodal inputs against defined questions. It does not create pictures or video: its role in a creative pipeline is classification and routing. The following integration is documentation-based and has not been executed by The Signal.

Where it fits in a creative workflow

A proposed use case is routing generated assets to an editorial team after generation. Start with a small set of clear labels and retain a human review path for ambiguous cases. Treat the model’s choice as a recommendation until your evaluation demonstrates acceptable error rates.

Prepare Workers AI access

  1. Identify your Cloudflare account ID and create a scoped token following the Workers AI REST setup.
  2. Set CLOUDFLARE_ACCOUNT_ID and CLOUDFLARE_AUTH_TOKEN in your local environment or server secrets.
  3. Run the Python example below. It uses only the standard library.
  4. For a video test, set REVIEW_VIDEO_DATA_URL to a valid data:video/mp4;base64,… value containing your own clip. Without that variable, the request evaluates text only.
import json, os, urllib.request

payload = {
    "model": "clef-omni",
    "state": "An uploaded product clip needs editorial review.",
    "questions": {
        "review_team": {
            "type": "choice",
            "instructions": "Which team should review this content?",
            "criteria": {
                "creative": "Composition or product presentation",
                "technical": "Broken media or playback",
                "unclear": "Insufficient evidence"
            }
        }
    }
}
# Optional: supply your own base64 data URL in a local environment variable.
if os.environ.get("REVIEW_VIDEO_DATA_URL"):
    payload["videos"] = [os.environ["REVIEW_VIDEO_DATA_URL"]]
url = ("https://api.cloudflare.com/client/v4/accounts/"
       + os.environ["CLOUDFLARE_ACCOUNT_ID"]
       + "/ai/run/@cf/cloudflare/clef-omni")
req = urllib.request.Request(url, data=json.dumps(payload).encode(), headers={
    "Authorization": "Bearer " + os.environ["CLOUDFLARE_AUTH_TOKEN"],
    "Content-Type": "application/json"
})
with urllib.request.urlopen(req, timeout=90) as response:
    print(response.read().decode())

This is an editorial adaptation of Cloudflare’s documented request schema, not a benchmark. Inspect the raw response and error handling before wiring it into automation. A test video must remain within the current model input limits.

Multimodal input is not an arbitrary file URL

The launch example passes embedded media through images, audio, and videos fields as base64 data URLs. Do not substitute an ordinary download link. Use small, non-sensitive test assets first and consult the current format and size requirements.

Pricing and license

The model page lists $0.15 per million input tokens at this check, with media converted to input tokens and no output-token charge. That is not a fixed price per video. Use returned usage and your billing dashboard to measure a real workload.

The official weights repository includes Apache-2.0. Downloadable weights and the hosted Workers AI service are separate choices; self-hosting still requires suitable hardware and a supported inference implementation. Our open-source AI models guide explains the wider selection context.

Test quality before choosing a model

Build a labeled evaluation set with ordinary cases, ambiguous cases, and poor-quality media. Keep examples used to refine your instructions separate from final evaluation examples. Compare per-label precision and recall, routing mistakes, failure rate, and cost per correctly handled item. Time the complete request, including encoding and upload; report median and p95 latency with sample count and media duration. Vendor speed figures are not a substitute for this test.

Use the same inputs and labels when comparing alternatives. Include a simpler baseline, such as existing rules or human routing, and define the error level that would make the system unsuitable before looking at results.

Official references

The Signal newsletter

Keep getting this

One edition a week on open models, local setups and the tools around them.

Read the latest issue

Email delivery opens once the newsletter platform is connected.