In my last post I talked about the LLM-as-a-Verifier paper and promised to explain the difference between their technique and ActiveGenie's.

In my last post I talked about the LLM-as-a-Verifier paper and promised to explain the difference between their technique and ActiveGenie's.

Both solve the exact same problem: when you ask an LLM to score two complex answers, it gives the same score to both. The paper measured this — 27% ties on Terminal-Bench V2 with a discrete scale. And a judge that ties ranks nothing.

The paper's solution: logprobs.

Instead of accepting the single score token the model spits out, they take the full probability distribution over the score tokens (on a 1-to-20 scale) and compute its expected value. The score stops being an integer and becomes continuous. Ties drop to zero, and just from that added granularity, accuracy goes from 73.1% to 77.5%.

ActiveGenie's solution: political debate.

Instead of asking for a score, I don't ask for a score at all. The two candidates enter a structured debate: "A" makes its case, "B" makes its case, "A" rebuts, "B" rebuts, each gives closing remarks, and an impartial judge decides. There's no tie because the format doesn't allow one — there's a winner and a loser.

And here's the trade-off nobody talks about.

Logprobs give a better signal: finer-grained, calibrated, and cheaper per comparison. There's just one catch — GPT-5.5 and Claude Opus don't expose logprobs. That means one extra call, extra latency, and a dependency on which provider exposes what.

The debate runs anywhere. It only needs text in, text out. That's exactly why ActiveGenie has been model-agnostic from day one: you swap providers in one config line and rewrite nothing. The price is a coarser signal — I know who won, but not by how much.

Except the two don't have to live apart.

In ActiveGenie's latest version I added logprobs support: if the chosen model exposes the distribution, the Comparator uses the continuous signal alongside the debate; if it doesn't, it falls back to the debate alone. You don't have to choose between a portable technique and a fine-grained signal — you use the best signal that model, on that call, can actually give you.

Which brings me back to the same point as always: the model matters less than the harness. With the right technique around it, you can run a smaller, faster, cheaper model and still deliver a consistent result.

The frontier isn't in the model. It's in what you build around it!