Fireworks says it serves over 40T tokens per day for customers' customized open models

Fireworks told its first company conference this week that it serves more than 40T tokens per day to customers' customized models. The company said this vertical grew over 2x since this summer. The figure covers only one part of Fireworks' inference business, so it is not a total for the company.

The term "customized" needs pinning down. Nathan Lambert, who wrote about the figure, said in a post on X that he would define these "specialized models" as "any post-trained model for a customer." Soumith Chintala added that AI-native companies such as Cursor and Decagon, which spread their load across Fireworks, Together and Baseten, would fall into that bucket and would make up a big chunk of the token numbers. Chintala's point is a guess about which customers produce the volume.

Lambert's broader analysis uses Baseten, Fireworks and Together AI as bellwethers for open model inference, along with the open model share of OpenRouter's token volume, which OpenRouter reports through its API. He said the hyperscalers probably have a large footprint too, through Amazon Bedrock and Azure's model garden, but he has not seen them break usage down by model category. He also wrote that all of these measurements are imprecise, citing noisy platform metrics, marketing-related figures from startups and imperfect public evaluations.

Lambert expects the next update to show a much higher total. Because the 40T number is a single slice that grew over 2x, he wrote that he would expect Fireworks' total to be "well above 100T" tokens by fall. That is his projection, not a number Fireworks has given. To put the startups' volumes in context, he plans to compare them with Google's, which the company reports for its entire platform and API usage in its earnings reports.