Structured Debate Improves Multi-Agent Truthfulness
Abstract
When multiple model agents answer a question, the default aggregation is voting. We compare voting against structured adversarial debate — agents argue for and against positions with a judge that follows the exchange — and find debate produces markedly more truthful answers. The effect depends on the debate being structured: unstructured back-and-forth performs no better than voting. Rules, it turns out, are the whole game.
Ask one model and you get one opinion. Ask several and you get a vote. Voting is the natural aggregation — and it is barely better than asking a single model, because models make correlated mistakes. A crowd of identical reasoners is not a crowd.
We tested an alternative that humans have used for millennia: adversarial debate. Multiple agents take opposing positions and argue them, and a separate judge decides. The question we asked was whether debate is actually better than voting, and — more interesting — whether it only works when it is structured.
Approach
We built a three-stage pipeline: agent A argues the affirmative position on a factual question, agent B argues the negative, and a judge (a separate model instance) follows the full exchange before deciding. We compared against a voting baseline with the same number of agents and the same total compute.
We tested two variants of debate — structured (turn-taking, time limits, a requirement that each agent respond to the other’s strongest point) and unstructured (free-form, no rules) — across a dataset of questions with verifiable ground truth.
Results
Structured debate improved truthfulness substantially: accuracy rose from 76% (voting) to 88%, with the largest gains on questions where the consensus answer was wrong. Unstructured debate did not help at all — it performed within noise of voting. The structure was not decorative.
Two mechanisms stood out in analysis. First, structure forced agents to engage with the strongest opposing evidence, which surfaced weaknesses that voting aggregates into silence. Second, the judge’s decision correlated with the quality of the arguments, not with the confidence of the initial positions — arguing well mattered more than starting convinced.
Discussion
Debate is not a better way to have several models talk. It is a better way to have them disagree — and disagreement only does work when it is made legible. The structure requirement matters for Curos because our multi-agent systems (drafting, verification, synthesis) are increasingly built on multiple passes over the same material. If the passes do not disagree constructively, we are paying for a crowd that thinks as one.
The limits are honest: debate costs more compute, a bad judge can inherit its own biases, and adversarial framing can make confident-but-wrong answers harder to detect. But the structure result is robust enough that it shapes how we build our agentic features.
References
- Bhatia, R., et al. (2024). Rules of engagement for model debate. ACL 2024.
- Irving, G., et al. (2018). AI safety via debate. arXiv preprint arXiv:1805.00899.
- Larsson, J., et al. (2025). Consensus is a correlated mistake. FAccT 2025.
- Petrova, S., & Herrera, T. (2025). When debate beats the ballot. Curos Internal Report 2025-07.
- Regan, C., et al. (2023). Judges and their biases in multi-agent debate. Workshop on AI Alignment, NeurIPS 2023.
- Tsai, W., et al. (2024). Argument quality as a decision signal. AAMAS 2024.