AI News

Artificial Analysis news

9 stories about Artificial Analysis.

ReportedModels ·

Artificial Analysis ranks Fable 5.2 first on its Intelligence Index at 65

Artificial Analysis scores Fable 5.2 at 65 on its Intelligence Index v5, ahead of Gemini 4 Carbon at 60 and Opus 5.5 at 59.

Why it matters: Independent index results help buyers compare frontier models, but the lead depends on effort setting and benchmark mix, so results need checking against their own workloads.

1 source
ReportedResearch · · Updated

Google DeepMind's MKA benchmark puts Claude Fable 5.2 first at 57% Climb WSR

Artificial Analysis and Google DeepMind launched MKA, a benchmark of 200 visual knowledge-work tasks, where Claude Fable 5.2 (max) leads with a 57% Climb WSR.

Why it matters: Cost per task varies nearly 30 times among models within about 10 points of the top score, so accuracy alone will not settle model choice for document-heavy knowledge work. Teams building visual document agents should weigh cost per task against performance.

2 sources
ReportedModels ·

Artificial Analysis says Meta's Muse Ember is the cheapest model scoring 36

Artificial Analysis says Muse Ember scores 36 on its Intelligence Index, matching Claude Haiku 5.5 (max) at about a third of the cost per task.

Why it matters: For agent workloads where cost per task drives the budget, Muse Ember offers a similar score at a fraction of the price of comparable models. Its low answer rate, though, means teams that rely on factual recall should expect more declined answers.

1 source
OfficialResearch ·

Artificial Analysis Endpoint Accuracy Index v1.1: 24 of 103 endpoints trail reference

Artificial Analysis says 24 of 103 serverless endpoints score statistically below its self-hosted reference in the updated Endpoint Accuracy Index v1.1.

Why it matters: Two endpoints serving the same open-weights model can differ noticeably in accuracy, so price and speed alone are not a sound basis for choosing a provider. According to the company, tool-calling and output-length settings are the most common sources of loss.

1 source
OfficialResearch ·

Artificial Analysis launches Decision Index v2 with calibration-weighted scoring

Artificial Analysis has released version 2 of its Decision Index, with 45% of the score now built on private datasets and a headline metric that rewards calibrated confidence.

Why it matters: Teams choosing decision models can see that cost per task and calibration can change rankings, and that per-question billing can make one model several times more expensive than another with similar list prices.

1 source
OfficialResearch ·

Artificial Analysis launches VIALS-AA; Gemini 4 Carbon (high) leads at 64%

Artificial Analysis launched VIALS-AA, a benchmark for scientific image reasoning, where Gemini 4 Carbon (high) leads with 64% of tasks answered correctly in all three repeats.

Why it matters: A single-pass score can hide large differences in reliability and cost on image-based scientific work, where a result that changes between runs can send teams toward wrong conclusions.

1 source
ReportedModels ·

Nanbeige releases open-weight 33B model scoring 20 on Artificial Analysis index

The dense 33B Nanbeige4.3-33B is released under Apache 2.0 and scores 20 on the Artificial Analysis Intelligence Index, level with Qwen3.8 27B.

Why it matters: Teams that want a small open model for local or self-hosted use now have another option at this score level, with a clear trade-off between terminal-use strength and factual recall. The higher token use per task also affects inference cost.

1 source
ReportedProducts ·

GitHub Copilot Fusion routes coding work between Anthropic and OpenAI models

GitHub Copilot Fusion switches coding tasks between Anthropic and OpenAI models mid-task instead of fixing developers to one model.

Why it matters: Teams standardizing on one coding model may need to test multi-model routing, since the reported cost per task is about half of a single-model setup.

1 source