Will an LLM report >50% score on ARC in 2025?
Plus
19
Ṁ3984Dec 31
85%
chance
1D
1W
1M
ALL
ARC is a general-purpose AI eval designed to test intelligence as opposed to memorization. https://arcprize.org/arc
This market resolves to Yes if there are public demonstrations of general-purpose base models from OpenAI, Anthropic or Deepmind that solve at least 50% of the ARC questions before 2026.
(note that this is separate from winning the arc price, which requires only using open-source models)
Get Ṁ1,000 play money
Related questions
Related questions
LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?
60% chance
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
94% chance
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
58% chance
Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?
66% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
60% chance
6 months from now will I judge that LLMs had already peaked by Nov 2024?
16% chance
What will be true of Anthropic's best LLM by EOY 2025?
Will RL work for LLMs "spill over" to the rest of RL by 2026?
40% chance
Will I write an academic paper using an LLM by 2030?
65% chance
Which company will have the best LLM by the end of 2024?