Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

EOY 2025: Will open LLMs perform at least as well as 50 Elo below closed-source LLMs on coding?

Will an LLM report >50% score on ARC in 2025?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

Will LLMs be able to formally verify non-trivial programs by the end of 2025?

Will an LLM agent complete >50% of the lab tasks on the Factorio Learning Environment benchmark in 2025?

LLM reaches >90% Brier score on Prophet Arena by 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

Will an LLM consistently create 5x5 word squares by 2026?