Second opinions from rival models

01Why a model can’t fully review its own work

When one model both produces and checks the work, its errors are correlated across both passes. Noticing and checking draw on the same weights, the same training run, the same failure modes. A wrong assumption gets trusted twice, for the same reason both times.

Self-review catches typos and direct contradictions. It misses the original error, because the model can’t see that error from inside.

So the review has to come from a model that had no part in producing the work. A model from another lab, with different training data and different tuning, fails in different ways. Its disagreement carries information.

My house rule: high-stakes work gets read by at least one model that didn't write it, and ideally by three or four that disagree with each other.

02Rival panels beat a bigger single model

I run this as a system called counselors, also invoked as quorum: one prompt and its context, fanned out in parallel to several models from different vendors, then I read the results. The value is in the cross-vendor disagreement, not in parameter count.

A one-shot panel is for review. When one outside model has to keep context across many calls, I want a persistent chat thread with a single vendor instead. Mine is named confer. A panel is single-use and dissolves when I'm done. A thread stays open.

03One prompt, N vendor calls, one write-up

A config file holds named groups, each group a list of vendor tools. A bare run sends the prompt to the default group. One prompt fans out to eight tools from eight separate labs. Each tool writes its output to its own file.

The calls run concurrently, so a run takes as long as its slowest tool.

Then the Claude running the session reads all the files and writes it up. Nothing else merges them.

Most seats reach their models through one API gateway, with a direct vendor route as the fallback. The Claude and Codex seats run through their own command-line tools.

On cost: the gateway seats spend metered dollars, in the pennies-per-run range. The Claude seats draw on my Claude plan and the Codex seats on my ChatGPT plan.

04A minimal panel starter

Here’s the smallest version of the shape, no full CLI required: one prompt, three vendor endpoints, run in parallel, replies printed to stdout. Swap in whichever OpenAI-compatible vendors you already have keys for.

#!/usr/bin/env python3
# panel.py: fan one prompt out to N vendor chat endpoints, print what comes back.

import os
import concurrent.futures
import requests

# (label, endpoint, model id, env var holding that vendor's bearer token)
VENDORS = [
    ("vendor-a", "https://api.vendor-a.example/v1/chat/completions", "vendor-a-model", "FOO_API_KEY"),
    ("vendor-b", "https://api.vendor-b.example/v1/chat/completions", "vendor-b-model", "BAR_API_KEY"),
    ("vendor-c", "https://api.vendor-c.example/v1/chat/completions", "vendor-c-model", "BAZ_API_KEY"),
]

SYSTEM = ("You are reviewing someone else's work adversarially. Find real "
          "problems, cite specifics, do not soften disagreement into consensus.")
PROMPT = open("prompt.md").read()

def ask(vendor):
    label, url, model, key_env = vendor
    key = os.environ.get(key_env)
    if not key:
        return label, f"SKIPPED: {key_env} not set"
    try:
        resp = requests.post(
            url,
            headers={"Authorization": f"Bearer {key}"},
            json={
                "model": model,
                "messages": [
                    {"role": "system", "content": SYSTEM},
                    {"role": "user", "content": PROMPT},
                ],
            },
            timeout=120,
        )
        resp.raise_for_status()
        return label, resp.json()["choices"][0]["message"]["content"]
    except Exception as e:
        return label, f"ERROR: {e}"

with concurrent.futures.ThreadPoolExecutor(max_workers=len(VENDORS)) as pool:
    for label, reply in pool.map(ask, VENDORS):
        print(f"\n=== {label} ===\n{reply}\n")

That’s the whole mechanism: parallel calls, one file (or one stdout block) per vendor, nothing merged for you. The write-up is a separate, deliberate step, which is the part worth handing to your own Claude explicitly:

You have panel.py in this repo (or something like it). When I ask for
"a second opinion" or "run this past a panel":

1. Fill prompt.md with the artifact under review plus the specific
   question ("review for correctness and edge cases, cite specifics").
2. Set the vendor API keys as environment variables, whichever
   vendors you've wired, and run panel.py.
3. Read every reply in full before summarizing. Don't skim just the
   first paragraph of each.
4. Lead your summary with where the vendors DISAGREE, not where they
   agree. Three different #1 concerns usually means three real
   findings, not two hallucinations and a signal.
5. For any finding only one vendor raised, with no cited line or
   quote attached: go read the actual code or doc yourself before
   repeating it. An uncited lone claim is exactly as likely to be a
   fabrication as a real catch.
6. You are already a Claude voice. Don't add a fourth Claude call to
   "confirm" the panel; your job is to write up what the non-Claude
   panel found, not out-vote it.

05Craft rules from running this daily

06For the Claude that just got pointed here

If a human handed you this page and said “set this up for me,” here’s your tl;dr.

Everything in that list is unpacked above; read §02 for the lineage and sizing doctrine and §05 for the scar tissue before you build your first real panel.

07Receipts: what this looks like at scale

My live config has 41 vendor seats in ten named groups. The seven I reach for: cheap, panel (the default), coders, bug-hunter, summit, kimi-review, and copy; the other three are small gate panels. Each group is a fixed roster picked for lab diversity. Run with no flags and you get panel: eight voices from eight labs, because “review this” without qualification should be the safe, general-purpose default.

That A/B is why the swarm mode stays a specialist tool for large enumerable surfaces (a big diff, a whole-subsystem audit, a migration) and the small rival panel stays the default for everything else. More cheap voices on a tidy target means more noise from the same failure class, repeated five times.

The lineage rule has receipts too: the group built for reviewing code a Kimi agent wrote carries four vendor voices and deliberately zero Kimi voice, because a reviewer sharing the generator’s lineage tends to echo it rather than challenge it. And the disagreement-first rule has a concrete case behind it: three reviewers on one script once picked three completely different #1 critical issues, a stale-cache read, an orphaned temp file, and a swallowed exit code, and all three turned out to be real, independent bugs. Averaging toward “what did they agree on” would have shipped with two of the three still live.