Will LLMs be worse than human level at forecasting when they are superhuman at most things?
➕
Plus
12
Ṁ421
2030
43%
chance

Resolves to my personal opinion, unless @SemioticRivalry disagrees.

Feel free to ask for clarification about what is meant in the comment but I'll likely keep it vague to resolve to the spirit of the question if precise definitions might take away from that

Get Ṁ1,000 play money
Sort by:

IMO forecasting is a broad and shallow skill rather than a narrow and deep one.

I think the canonical example of a narrow and deep skill is chess:

  • For the most part humans who are good at chess have to spend decades practicing chess specifically (to the point where if you start playing later than childhood, it's empirically almost impossible to reach the top tier of performance)

  • I don't believe the component skills of chess are particularly transferable to other domains on the margin (over and above the effect that you have to be fairly bright to get real good at chess)


In contrast, forecasting:

  • You regularly see people drop in to the field with a broad technical background and be in the 95th+ percentile with <1y of forecasting-specific effort

  • the main component skills that make a good forecaster are mostly pretty transferable e.g.

    • mastery of basic coding

    • internet street smarts re: digging up sources of data

    • detail-oriented communication style

    • mastery of high-school level stats

    • not having completely debilitating biases/brainworms

I think that this is an argument that we shouldn't expect AI forecasting to particularly lag other domains: it'll master the component pieces for their general utility and there won't be much 'forecasting specific' left over to make a difference.

Does any amount past median human level count as superhuman?

@toms I think superhuman is generally better than (almost?) any human, so no

bought Ṁ10 YES

We die in most timelines where this resolves no

@Joshua We die in most timelines ̶w̶̶h̶̶e̶̶r̶̶e̶̶ ̶̶t̶̶h̶̶i̶̶s̶̶ ̶̶r̶̶e̶̶s̶̶o̶̶l̶̶v̶̶e̶̶s̶̶ ̶̶n̶̶o̶̶