Blog · 2 posts, newest first

Results, runs and how we measured them.

Every post ends with the numbers and how to reproduce them.

Subscribe by RSS

Featured Benchmarks

All of MMLU in one request.

All 14,042 questions, about 1.5M tokens, sent as one API request. How we ran it, how long it took, and how it compares with other leading models.

Read the post
Questions
14,042
Requests
1
Wall-clock
[X] s
Accuracy
[X]%
esc
↑↓ move↵ open⌘K