Gemini 4 Carbon beats Argon in new tests, with time and cost trade-offs
Independent evaluations give Gemini 4 Carbon more substance than the early demos: Zapier’s automation test shows a modest gain in completed workflows, while Agent Arena ranks it above Argon but reports longer median task times and higher costs. The results support stronger capabilities, not a claim that Carbon can reliably work unattended.
The actual story.
Our earlier coverage of tester impressions described selected 3D demonstrations and reports of slower responses. There is now firmer evidence: Google has named the model, and outside evaluators have published results. Those tests do not establish that every 3D or coding workflow improves.
In Zapier’s AutomationBench 1.0.6, Carbon at high effort passed 354 of 657 tasks, or 53.9%, against Argon’s 51.3% at high effort—a 2.6-percentage-point gain. Carbon’s run took place October 26–27. Tasks use simulated business apps; a pass requires every check on the final state to succeed. Agents cannot ask clarifying questions or receive human help. A failed task does not necessarily mean a harmful outcome.
More effort has a price
Carbon’s medium setting passed 53.0% at an estimated $1.65 per task, compared with 53.9% at $2.12 on high. Zapier estimates those costs using $4/$20 per million input/output tokens and a cache assumption; they are not bills or estimates at Google’s introductory rates. The small extra gain makes effort level a practical choice, not simply a setting to maximize.
Agent Arena’s October 28 snapshot places Carbon second, provisionally, versus Argon seventh. Across 26,511 Carbon sessions, its median task time was 11.4 minutes versus Argon’s 7.5, and median cost was $1.77 versus $1.05. These are platform-session results, not a matched test of every reader’s workload. Arena’s 16.04% “net improvement” is a ranking signal, not a task-success rate; its methodology differs from Zapier’s strict pass/fail test.
Artificial Analysis also places Carbon first on its Intelligence Index at 59. That strengthens the case for evaluating it, but an aggregate score cannot answer whether it finishes your work correctly.
Access remains limited. Google’s October 28 announcement starts with selected cyber defenders, then additional trusted partners, without a general-access date. Google announced introductory pricing of $2/$10 per million input/output tokens. Keep that offer separate from benchmark cost assumptions. Its claims about reduced fabrication and secure coding include internal evaluations that MDN has not independently tested.
Carbon’s stronger results come with a trade-off: more time and higher cost per task. That can be worthwhile for difficult work if it saves enough correction and rework afterward. For routine tasks, a faster, cheaper model may still be the better choice. The question is whether the extra capability improves the finished job enough to justify what you spend.
Receipts, not rumors.
- Artificial Analysis
- Previous MDN story
- Zapier AutomationBench: Carbon results and methodology
- Agent Arena leaderboard
- Agent Arena methodology
Read the original reporting: Google; Artificial Analysis
The reporting above provides context. Illustrations are editorial interpretations, not source photographs.
What’s your take?