Will a text model achieve 100% performance on the MMLU in five years?
Mini
3
Ṁ402028
28%
chance
1D
1W
1M
ALL
This video predicts that a model will achieve 100% performance on the Massive Multitask Language Understanding Benchmark.
This will resolve YES if a model is listed as having achieved 100 percent performance on the MMLU on papers with code, in a technical document, or published article by 3:00 PM EST June 28, 2028. This will resolve NO if a model is not listed as ahving achieved 100 percent performance on the MMLU.
Get Ṁ1,000 play money
Related questions
Related questions
By 2025 will there be a competitive large language model with >50% of the total training data generated from a large language model?
75% chance
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
58% chance
Who will have the best Text-to-Image Model at the end of 2025 (as decided by the Artificial Analysis Leaderboard)?
Will an open-source text-to-music LLM model be able to create an entire album as convincing as a human by end of 2025?
26% chance
Who will have the best Text-to-Video Model at the end of 2025 (as decided by the Artificial Analysis Leaderboard)?
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.
68% chance
Benchmark Gap #5: Once a single AI model solves >= 95% of miniF2F, MATH, and MMLU STEM, will it be less than two years before AI models are used as entry-level data science / data analysis / statistics workers?
67% chance
Will a team consisting primarily of ML models complete MIT Mystery Hunt by 2030?
45% chance