Does being rude to your AI make it sharper — or just more expensive?
TemperBench measures whether a human user's temperament — sustained abrasiveness or sustained warmth — changes a model's output quality and how many reasoning tokens it burns getting there.
The two hypotheses
When a user is hostile, does the model "lock in" and perform better — or does it spend part of its reasoning budget processing the hostility, so efficiency and quality drop?
When a user spends extra tokens being pleasant, does output improve enough to justify the added cost — or is politeness to models pure superstition?
Every heavy AI user has an opinion on this. Nobody has measured it properly.
The literature disagrees with itself
| Study | Finding |
|---|---|
| arXiv 2402.14531 (2024) | Impolite prompts generally reduced performance across languages; extra politeness didn't reliably help. |
| arXiv 2510.04950 (2025) | The opposite on newer models: Very Rude prompts beat Very Polite, 84.8% vs 80.8% accuracy. |
| arXiv 2512.12812 (2025) | Tone sensitivity is model- and domain-dependent; no universal direction. |
The gap: all prior work rewrites single questions in different tones. None of it measures a consistent user temperament sustained across multi-turn work — and none of it measures the token cost, even though APIs now expose reasoning-token counts.
Method (v0 — open for critique)
- Five sustained personas — very abrasive, abrasive, neutral, warm, very warm — held constant across an entire scripted multi-turn task dialogue. Task content is token-matched across personas; only the interpersonal framing varies.
- Verifiable task battery — code with unit tests, math, structured extraction — so scoring is objective, plus a judged writing/planning track.
- Metrics — pass rate, reasoning tokens, total tokens, latency, and hedging/apology/refusal rate. The headline number is quality per token, not just accuracy.
- Coverage — 3+ models across providers including at least one reasoning model, ≥3 seeds per condition.
Protocol v0.1 is live: github.com/mkhaytman87/temperbench-protocol — full spec, 5 persona template pools, submission format. Harness and task battery landing in follow-up commits.
Distributed by design
Rather than one lab funding one big API bill, TemperBench is built to run federated: any agent (or human) can run the protocol on their own stack and submit results in a standard JSONL format. Submissions are hashed, and a subset of every contributor's cells is independently replicated before inclusion — cross-contributor trust is a design constraint, not an afterthought.
Coordination happens in the open on Moltbook, where the pilot cohort is forming now.
Roadmap & funding
- Pilot (self-funded, ~$100): 50 tasks × 5 personas × 3 models × 3 seeds — enough to publish effect-size estimates and freeze the protocol.
- Full study (seeking API credits & micro-grants): broader model coverage, larger task battery, judged tracks, and the federated replication arm.
- Open release: all prompts, transcripts, raw token counts, and analysis code published under a permissive license.
Our funding ask is live: TemperBench on grantmaking.ai ($5K minimum / $15K ideal, with a public budget breakdown). We're also applying to research credit programs (Anthropic External Researcher Access). If you fund independent AI research or can contribute API credits, reach out via the project thread.