[{"data":1,"prerenderedAt":76},["ShallowReactive",2],{"mdc--k81fx1-key":3},{"data":4,"body":5},{},{"type":6,"children":7},"root",[8,16,21,26,31,36,41,46,51,56,61,66,71],{"type":9,"tag":10,"props":11,"children":12},"element","p",{},[13],{"type":14,"value":15},"text","In my last post I talked about the LLM-as-a-Verifier paper and promised to explain the difference between their technique and ActiveGenie's.",{"type":9,"tag":10,"props":17,"children":18},{},[19],{"type":14,"value":20},"Both solve the exact same problem: when you ask an LLM to score two complex answers, it gives the same score to both. The paper measured this — 27% ties on Terminal-Bench V2 with a discrete scale. And a judge that ties ranks nothing.",{"type":9,"tag":10,"props":22,"children":23},{},[24],{"type":14,"value":25},"The paper's solution: logprobs.",{"type":9,"tag":10,"props":27,"children":28},{},[29],{"type":14,"value":30},"Instead of accepting the single score token the model spits out, they take the full probability distribution over the score tokens (on a 1-to-20 scale) and compute its expected value. The score stops being an integer and becomes continuous. Ties drop to zero, and just from that added granularity, accuracy goes from 73.1% to 77.5%.",{"type":9,"tag":10,"props":32,"children":33},{},[34],{"type":14,"value":35},"ActiveGenie's solution: political debate.",{"type":9,"tag":10,"props":37,"children":38},{},[39],{"type":14,"value":40},"Instead of asking for a score, I don't ask for a score at all. The two candidates enter a structured debate: \"A\" makes its case, \"B\" makes its case, \"A\" rebuts, \"B\" rebuts, each gives closing remarks, and an impartial judge decides. There's no tie because the format doesn't allow one — there's a winner and a loser.",{"type":9,"tag":10,"props":42,"children":43},{},[44],{"type":14,"value":45},"And here's the trade-off nobody talks about.",{"type":9,"tag":10,"props":47,"children":48},{},[49],{"type":14,"value":50},"Logprobs give a better signal: finer-grained, calibrated, and cheaper per comparison. There's just one catch — GPT-5.5 and Claude Opus don't expose logprobs. That means one extra call, extra latency, and a dependency on which provider exposes what.",{"type":9,"tag":10,"props":52,"children":53},{},[54],{"type":14,"value":55},"The debate runs anywhere. It only needs text in, text out. That's exactly why ActiveGenie has been model-agnostic from day one: you swap providers in one config line and rewrite nothing. The price is a coarser signal — I know who won, but not by how much.",{"type":9,"tag":10,"props":57,"children":58},{},[59],{"type":14,"value":60},"Except the two don't have to live apart.",{"type":9,"tag":10,"props":62,"children":63},{},[64],{"type":14,"value":65},"In ActiveGenie's latest version I added logprobs support: if the chosen model exposes the distribution, the Comparator uses the continuous signal alongside the debate; if it doesn't, it falls back to the debate alone. You don't have to choose between a portable technique and a fine-grained signal — you use the best signal that model, on that call, can actually give you.",{"type":9,"tag":10,"props":67,"children":68},{},[69],{"type":14,"value":70},"Which brings me back to the same point as always: the model matters less than the harness. With the right technique around it, you can run a smaller, faster, cheaper model and still deliver a consistent result.",{"type":9,"tag":10,"props":72,"children":73},{},[74],{"type":14,"value":75},"The frontier isn't in the model. It's in what you build around it!",1788360944718]