Carnegie Mellon pre-print finds Jev fast but weak on math and logic judgments
A pre-print co-authored by Carnegie Mellon's Ramayya Krishnan reports Jev has a 150 ms median latency but accuracy drops on math and logic judgments.
Why it matters: The results suggest decision models can cut latency and cost for well-defined judgments, but they need checks before handling math or logic-heavy decisions. A hybrid design that escalates uncertain cases to an LLM is one tested option.