Blog

Results, runs and how we measured them. Every post ends with the numbers and how to reproduce them.

Featured 24 Sep 2026 Benchmarks

All of MMLU in one request.

All 14,042 questions, about 1.5M tokens, sent as one API request. How we ran it, how long it took, and how it compares with other leading models.

Questions
14,042
Requests
1
Wall-clock
[X] s
Accuracy
[X]%
24 Sep 2026 Benchmarks

All of MMLU in one request.

[X] s · [X]% 24 Sep 2026 Benchmarks

Pip vs three models. Four benchmarks.

[X]× faster · [X]/4 most accurate