Templates, one-pass readout, honest uncertainty and routing for a zero-shot decision engine on DiffusionGemma.
dgem takes some text or an image (the state) and a list of typed questions from a template file. For each question it returns the answer it picked, a probability for every allowed answer, and a hesitation score. All questions are answered together in one forward pass. We don't change the model weights. Confident answers are used directly; hesitant ones go to a person or to Gemini.
This page explains what you can build, how a decision is read, what the confidence numbers mean and what we learned testing ways to correct them. The documentation starts at the docs home.
dgem serve → localhost:8090
Behaviour lives in a template file (.json.tmpl) in git. Adding a question or an option is an edit, not a training job.
boolean, choice (≤26 options), scoreQuestions that depend on another answer (depends_on) run in a second pass; questions with more than 26 options use a two-stage bracket.
Unlike a chat model, nothing is written word by word, so there is no JSON to parse and the probabilities belong to the decision itself rather than to formatting tokens.
Hesitation is the entropy of an answer's probabilities, scaled to 0–100% by the number of options: 0% when one answer has all the probability, 100% for a coin toss. It makes a yes/no question and a 26-option question comparable.
| Hesitation | Meaning | What to do |
|---|---|---|
| < 16% | Clear | Act automatically |
| 16–50% | Somewhat unsure | Escalate if errors are costly |
| > 50% | Very unsure | Send to a person or to Gemini |
In our tests hesitation alone flags most wrong answers. Calibration is still policy- and data-specific: check your own with Calibrate your policy, and read Confidence beyond Shannon for how hesitation-gating works.
DiffusionGemma, like other language models, favours the first option when it is unsure. We tested ways to fix that at request time; dgem now gates on hesitation and monitors option order on every release instead:
| Technique | Finding | Status |
|---|---|---|
| Null-prior de-biasing ("tare the scale") | Helped some suites and hurt others | Research; validate per suite |
| Reversed "mirror" ballot on the same canvas | Lost accuracy because both slots used letters A, B, C for different options; digit labels fix that but add little | Research |
| Mirror in a separate pass | A small extra signal, no better than reading twice, for double the cost | Research |
| Temperature scaling (held-out, per policy) | Improves calibration when fitted on the policy's own data; the best value differs by domain | Recommended per policy |
| Hesitation gate + Gemini cascade | Recovers most of the larger model's accuracy while escalating a minority of items | Recommended |
…mirror… changed answers. Use neutral, descriptive ids. And don't give two slots the same letters for different meanings.diffgemma, 4-bit): development and offline use.A gateway (dgem serve) routes to Vertex first and falls back to Cloud Run. The same image, templates, CLI, HTTP API and MCP server work on a laptop, Cloud Run and Vertex. /health reports the serving version. Measure latency on your own deployment: Latency and capacity.
make build
./bin/dgem decide -t templates/support_triage.json.tmpl \
-v 'ticket=Charged twice for the annual plan' --stats
# or connect an MCP client to your own gateway:
./bin/dgem mcp --remote https://<your-dgem-gateway>/mcp
Tutorial: Your first decision policy. Deploying your own: From laptop to production. Changes: CHANGELOG.