Marin finishes main pretraining of 535B Mochi model on 18T tokens
Marin says its Mochi 535B-A23B model finished main pretraining on 18T tokens, with long-context training and fine-tuning next.

Watch the short
Under a minute, read by an AI anchor. Made .
YouTube loads only when you press play.
A Marin team member, dlwh, said the Mochi 535B-A23B model has finished main pretraining on 18T tokens. The next stages are long-context training and supervised fine-tuning (SFT), the step that teaches a base model to follow instructions from curated examples. dlwh said checkpoints are hoped for the following week, which is a hope rather than a commitment.
What changed
The Mochi 535B-A23B model has finished main pretraining on 18T tokens, with long-context training and supervised fine-tuning next.
Type: Other · Applies to: Mochi 535B-A23B
Why it matters
A 535B-parameter model with 23B active parameters is another large open-weight candidate in development. No weights or evaluations are out yet.
Original source
We have no link to the original source for this story yet.
Where we saw it
Latent Space
AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works