Will Matthew and Tamay win their bet on longer AI timelines by 2026?
➕
Plus
16
Ṁ604
2026
9%
chance
From https://www.lesswrong.com/posts/X3p8mxE5dHYDZNxCm/a-concrete-bet-offer-to-those-with-short-ai-timelines @MatthewBarnett and Tamay believe that AI timelines are longer than expected. Specifically, they were offering "1:1 odds that no more than one of the following events will occur by 2026-01-01: - A model/ensemble of models achieves >80% on all tasks in the MMLU benchmark - A credible estimate reveals that an AI lab deployed EITHER >10^30 FLOPs OR hardware that would cost $1bn if purchased through competitive cloud computing vendors at the time on a training run to develop a single ML model (excluding autonomous driving efforts) - A model/ensemble of models will achieve >90% on the MATH dataset using a no-calculator rule - A model/ensemble of models achieves >80% top-1 strict accuracy on competition-level problems on the APPS benchmark - A gold medal for the IMO Grand Challenge (conditional on it being clear that the questions were not in the training set) - A robot that can, from beginning to end, reliably wash dishes, take them out of an ordinary dishwasher and stack them into a cabinet, without breaking any dishes, and at a comparable speed to humans (<120% the average time) - Tesla’s full-self-driving capability makes fewer than one major mistake per 100,000 miles" Resolves YES if they win their bet (no more than one of the above occurs by the end of 2025).
#AI
#Technical AI Timelines
Get Ṁ1,000 play money
1 Comment
Sort by:

I just wrote a retrospective on the bet. I think it's highly likely that we'll lose, unless something weird happens, like people stop publishing on all the benchmarks that we used for the bet. I think what happened is that (1) my AI timelines were simply too long, and (2) I wasn't thinking very carefully about the milestones, and didn't realize that extrapolating past trends reveals that we are expected to lose unless progress slows down by a lot.

Related questions