AI News
All stories
Reported

Theo says Gemini 4 Carbon runs about five times slower than other models in his work

Commentator Theo says his own audit found Gemini 4 Carbon about five times slower than comparable models, with over 11 seconds before responses start.

Models · · 2 min read

Commentator Theo of the t3.gg channel says Gemini 4 Carbon is about five times slower than comparable models in his day-to-day work. He places much of the problem before a request reaches the model. In a video titled "I was wrong...", he says he had understated the speed gap in his earlier review. He audited his real-world usage across models and found that, on like-for-like work, Gemini was consistently about five times slower than the others.

The central claim concerns time to first response. Theo says the model consistently takes over 11 seconds from the moment a request reaches Google's server until it starts doing anything. He links this to multi-step agent turns. The model needs the whole conversation history on each step, so every step adds overhead as the history is sent up, parsed, matched to a cache, prefilled and loaded before output begins. An 11-second delay per step compounds over a multi-step turn. Theo argues that Artificial Analysis latency figures understate this, because they measure from the moment the server accepts a request to the response, and many of their tasks are not agentic or multi-turn.

Theo's evidence in the video is mostly his own usage and a live demo. When Gemini was brought in as a judge for work nine times, he says the parent stopped the agents' threads after the first three. He then updated his agent skill to tell agents to stop waiting on Gemini when they have work to fix, which means his numbers understate the delay. In a live demo of a single step, Gemini finished last after another model ran a slow tool call. He has not measured the full extent of the slowdown, because doing so would have slowed his regular work.

Theo places part of the blame on caching and hosting. He says the model reads a file eight to sixteen times on average. He also says Gemini writes to a cache and then fails to read the entry it just wrote about 30 percent of the time, sometimes reading an older entry instead. On hosting, he says Vertex AI is unusable for this model, and that the workable route runs through endpoints he describes as going against the terms of service. He further objects that reasoning effort is encoded in the model name rather than as a separate field, which he says breaks analytics.

Theo also points to cost. He says the subscription price doubles on December 1. He acknowledges that some people have argued subscriptions are slower than the API, and says a reply to him pointed to the opposite.

Theo quotes a reply from Verun, who went to DeepMind. Verun said the team is looking into how to optimize this, and noted that there is always a trade-off between latency and cost but that they should do better. The claims about speed, caching and hosting are Theo's own observations. He says he has not published a full measurement, so they have not been independently verified.

What changed

Type: Other · Applies to: Gemini 4 Carbon

Why it matters

Teams running multi-step coding or agent workflows may find that end-to-end turn time, not token speed, dominates cost and productivity. If the claimed per-turn delay holds, it could matter more than benchmark rankings.

What you can do

Measure end-to-end time per agent turn, including time to first token, on your own workloads before choosing Gemini 4 Carbon for production agents.

Original source

We have no link to the original source for this story yet.

Where we saw it

1 source

Theo - t3.gg

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works