Most gateways route a request to one upstream. That's the right default for throughput and cost. But for a narrow class of prompts — where a wrong answer is expensive — a single model is a single point of failure. Referee mode is MindBalancer's answer to that.
What it does
Referee mode fans one prompt out to several providers in parallel, collects their answers, and passes them to a referee model that synthesizes a single cross-validated response. You configure the minimum number of successful responses required, which providers to query, the referee model, and a timeout.
{
"model": "referee",
"referee": {
"min_responses": 2,
"referee_model": "gpt-4o",
"timeout_ms": 20000
},
"messages": [{ "role": "user", "content": "..." }]
}Transparency: referee_info
The response includes a referee_info object so the synthesis is never a black box — how many providers were queried, how many succeeded, and the latency. You can log it, surface it to reviewers, or gate on it.
When to use it (and when not to)
- Good fit: financial, medical or legal content where a hallucination is costly and latency budget is generous.
- Good fit: extraction or classification you want a second and third opinion on before it's trusted.
- Poor fit: high-throughput chat where cost and latency dominate — use normal routing there.
- Poor fit: anything where you only have one capable provider configured.
How it fits the rest
Referee mode rides on the same infrastructure as everything else: providers live in ai_servers, you administer it over SQL, and Prometheus exposes referee_* metrics next to your request, token and cache metrics.