AI News
All stories
Reported

Marin finishes main pretraining of 535B Mochi model on 18T tokens

Marin says its Mochi 535B-A23B model finished main pretraining on 18T tokens, with long-context training and fine-tuning next.

Models · · 1 min read

Watch the short

Under a minute, read by an AI anchor. Made .

YouTube loads only when you press play.

A Marin team member, dlwh, said the Mochi 535B-A23B model has finished main pretraining on 18T tokens. The next stages are long-context training and supervised fine-tuning (SFT), the step that teaches a base model to follow instructions from curated examples. dlwh said checkpoints are hoped for the following week, which is a hope rather than a commitment.

What changed

The Mochi 535B-A23B model has finished main pretraining on 18T tokens, with long-context training and supervised fine-tuning next.

Type: Other · Applies to: Mochi 535B-A23B

Why it matters

A 535B-parameter model with 23B active parameters is another large open-weight candidate in development. No weights or evaluations are out yet.

Original source

We have no link to the original source for this story yet.

Where we saw it

1 source

Latent Space

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works