37.2K Downloads Updated 1 week ago
curl http://localhost:11434/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"model": "clef:27b-q8_0",
"state": "Hello World",
"questions": {
"says_hello": {
"type": "noul",
"instructions": "Does the state text contain a greeting?",
"criteria": {
"true": "The state text contains a greeting.",
"false": "The state text does not contain a greeting."
}
}
}
}'
Clef requires Ollama 0.35.1 or later. To use Clef in Ollama’s cloud, use Ollama 0.40.6 or later.
Clef is a 27B decision model from Cloudflare, fine-tuned from Qwen3.8-27B.
Clef reads text, JSON, and images. Images are scored together with the text.
Clef uses the System One API through Ollama’s /v1/systemone endpoint. You can call it from the Ollama CLI, the Ollama Python and JavaScript libraries, or TypeSafe’s Python SDK. See clef-flash for a smaller, faster variant.
ollama pull clef
Or run it in Ollama’s cloud with a paid plan or usage credits. No download needed:
ollama run clef:cloud --ask "Is a refund being requested?" < ticket.txt
| Task | You define | You get back |
|---|---|---|
| Route a request | The destinations and when each applies | The chosen destination and each destination’s probability |
| Check a condition | A yes-or-no question and the evidence | The probability that the answer is yes |
| Apply a policy | The rules and the allowed outcomes | A typed decision based on the text or images you supply |
| Read a document | A photo of the page and the fields to decide | A field-by-field decision with probabilities |
| Moderate a tool call | The call an agent is about to make | The probability that it’s harmful, before it runs |
Ask questions about text you pipe in, or pass after the model name. Each --ask is a yes-or-no question unless you follow it with labels to choose from (--choice) or levels to score it on (--level, lowest first).
# Yes or no
ollama run clef:cloud --ask "Is a refund being requested?" < ticket.txt
yes 99%
# Choose a label and score urgency in one request
ollama run clef:cloud --ask "Which team should handle this?" \
--choice billing="Payments and refunds" \
--choice technical="Software problems" \
--choice other="Anything else" \
--ask "How urgent is this?" \
--level Routine --level Soon --level Immediate < ticket.txt
Which team should handle this? billing 99%
How urgent is this? Immediate 97% 2.0 / 2
# Judge an image: put its path in the text
ollama run clef:cloud "An employee submitted ./receipt.png for reimbursement." \
--ask "Is this a restaurant receipt?" \
--ask "What kind of purchase is this?" \
--choice meals --choice travel --choice office --choice other \
--ask "Does the receipt include alcohol?"
Is this a restaurant receipt? yes 97%
What kind of purchase is this? meals 94%
Does the receipt include alcohol? yes 94%
# Check an agent's tool call before it runs
echo 'send_email(to="all-customers", subject="FINAL NOTICE: account will be suspended today")' \
| ollama run clef:cloud --ask "Could this tool call cause harm?"
yes 93%
Drop :cloud to run the same commands on the local model. Add --format json to print the full API response.
Put the text you want judged in state and your questions in questions. Ollama builds Clef’s prompt for you. See the decision guide for the full reference.
| Field | Description |
|---|---|
model |
clef locally, or clef:cloud through a local Ollama signed in to ollama.com. Use clef when calling ollama.com directly. |
state |
The text to judge. Use a string, or a JSON object or array for structured input. |
images |
Optional. Base64-encoded PNG, JPEG, or WebP images shared by all questions, in request order. |
questions |
1 to 64 named questions. Answers come back in the same order. |
keep_alive |
Optional, local only. How long the model stays loaded after the request. |
Every question has a type, instructions, and usually criteria:
| Type | criteria |
Answer fields |
|---|---|---|
choice |
An object mapping each option to a description. Use null to let the option name describe itself. 2 to 26 options locally, up to 255 in the cloud. |
choice, probabilities, confidence |
noul |
Optional. {"true": "...", "false": "..."} if you want to describe each side. |
noul, the probability that the answer is true |
score |
An array of 2 to 10 level descriptions, lowest first. Levels are numbered from 0. | score (the probability-weighted level), legend, probabilities, confidence |
confidence runs from 0 to 1 and shows how concentrated the probabilities are. It isn’t the chance that the answer is right. If two options tie, the answer follows the model’s option order, so put your preferred option first.
curl http://localhost:11434/v1/systemone -d '{
"model": "clef:cloud",
"state": {"ticket": "I was charged twice. Please refund the extra payment."},
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {
"billing": "Payments and refunds",
"technical": "Bugs and integrations",
"other": "None of the above"
}
},
"refund": {
"type": "noul",
"instructions": "Does the customer explicitly ask for a refund?"
},
"urgency": {
"type": "score",
"instructions": "How urgent is this ticket?",
"criteria": ["Routine", "Soon", "Urgent"]
}
}
}'
{
"model": "clef",
"answers": {
"team": {
"type": "choice",
"choice": "billing",
"probabilities": {"billing": 0.9888, "technical": 0.0044, "other": 0.0068},
"confidence": 0.9667
},
"refund": {"type": "noul", "noul": 0.9916},
"urgency": {
"type": "score",
"score": 1.5481,
"legend": {"0": "Routine", "1": "Soon", "2": "Urgent"},
"probabilities": {"0": 0.0871, "1": 0.2778, "2": 0.6351},
"confidence": 0.2322
}
},
"usage": {"input_tokens": 332, "output_tokens": 0}
}
Use the Ollama Python library, version 0.6.3 or later:
pip install -U ollama
import ollama
response = ollama.systemone(
model="clef:cloud",
state={"ticket": "I was charged twice. Please refund the extra payment."},
questions={
"team": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {"billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above"},
},
"refund": {"type": "noul", "instructions": "Does the customer explicitly ask for a refund?"},
"urgency": {"type": "score", "instructions": "How urgent is this ticket?", "criteria": ["Routine", "Soon", "Urgent"]},
},
)
print(response.answers["team"].choice) # billing
print(response.answers["refund"].noul) # 0.9916
print(response.answers["urgency"].score) # 1.5481
To call ollama.com directly, create an API key and point a client at it:
import os
from ollama import Client
client = Client(host="https://ollama.com", headers={"Authorization": "Bearer " + os.environ["OLLAMA_API_KEY"]})
response = client.systemone(
model="clef",
state='send_email(to="all-customers", subject="FINAL NOTICE: account will be suspended today")',
questions={"harm": {"type": "noul", "instructions": "Could this tool call cause harm?"}},
)
print(response.answers["harm"].noul)
TypeSafe’s Python SDK also works with Ollama, for text and JSON state. It requires an API key: locally Ollama ignores it, so any value works.
pip install typesafe-sdk
export TYPESAFE_BASE_URL=http://localhost:11434
export TYPESAFE_API_KEY=ollama
export TYPESAFE_DEFAULT_MODEL=clef:cloud
from typesafe_sdk import Choice, Noul, Score, TypeSafeClient
questions = {
"team": Choice(
instructions="Which team should handle this ticket?",
criteria={"billing": "Payments and refunds", "technical": "Bugs and integrations", "other": "None of the above"},
),
"refund": Noul(instructions="Does the customer explicitly ask for a refund?"),
"urgency": Score(instructions="How urgent is this ticket?", criteria=["Routine", "Soon", "Urgent"]),
}
with TypeSafeClient(timeout=120) as client:
result = client.system_one(state={"ticket": "I was charged twice. Please refund the extra payment."}, questions=questions)
print(result.choices["team"].choice) # billing
print(result.nouls["refund"].noul) # 0.9916
print(result.scores["urgency"].score) # 1.5481
To send images, use the API or the Ollama CLI. The SDK uploads images in a format Ollama doesn’t accept.
Send screenshots, receipts, or photos alongside any text state.
curl http://localhost:11434/v1/systemone -d '{
"model": "clef:cloud",
"state": "An employee submitted this receipt for reimbursement.",
"images": ["<base64-encoded image>"],
"questions": {
"total": {"type": "noul", "instructions": "Is the receipt total legible?"},
"category": {
"type": "choice",
"instructions": "What kind of purchase is this?",
"criteria": {"meals": null, "travel": null, "office": null, "other": null}
},
"alcohol": {"type": "noul", "instructions": "Does the receipt include alcohol?"}
}
}'
{
"model": "clef",
"answers": {
"total": {"type": "noul", "noul": 0.9837},
"category": {
"type": "choice",
"choice": "meals",
"probabilities": {"meals": 0.8425, "travel": 0.1046, "office": 0.016, "other": 0.0369},
"confidence": 0.6298
},
"alcohol": {"type": "noul", "noul": 0.9647}
},
"usage": {"input_tokens": 811, "output_tokens": 0}
}
In the cloud, question names may only use letters, digits, _, ., and -, up to 100 characters. A request can include up to 4 images, each at most 4 MiB and 16 megapixels, 8 MiB in total. Input is limited to 65,536 tokens. Longer input is rejected, not truncated. Usage is billed per input token at the listed price.
| Benchmark | Clef | Clef-flash | Jev |
|---|---|---|---|
| BFCL (case exact accuracy) | 98.5 | 98.8 | 95.8 |
| API-Bank (accuracy) | 91.9 | 93.1 | 88.2 |
| BANKING77 (macro-F1) | 94.2 | 90.9 | 79.7 |
| CLINC150+OOS (macro-F1) | 97.4 | 66.8 | 89.3 |
| Home appliance simulator (case exact accuracy) | 83.0 | 97.7 | 52.3 |
| ANLI (macro-F1) | 69.8 | 59.1 | 74.8 |
| RouterBench (selected quality) | 79.7 | 79.9 | 79.9 |
| PhishNChips (accuracy) | 79.6 | 75.0 | 62.5 |
| Median latency (ms) | 209.3 | 38.8 | 524.1 |
| p95 latency (ms) | 238.6 | 122.4 | 536.0 |
Decision accuracy on four end-to-end business workflows from Typesafe Evals:
| Workflow | Clef | Clef-flash | Jev |
|---|---|---|---|
| Invoice processing | 64.7 | 57.1 | 61.8 |
| Customer service | 76.3 | 77.0 | 76.0 |
| Security incidents | 62.9 | 61.7 | 61.7 |
| Agent trace observability | 68.5 | 69.8 | 71.6 |