What will be true of Gemini 2?
Mini
21
Ṁ1313
2027
99%
It will be released in 2024
99%
It will be released before GPT-5
72%
>20% (5-shot) on SWEBench (current sota ~ 6%?)
59%
It will get into at least one PR blunder, like the woke issue for Gemini 1 Ultra
58%
It will score better than o1-preview at AIME (44.6% in single pass)
54%
It (the strongest variant) will be better than the best OpenAI model at the moment of release
42%
It will score >70% on GPQA (diamond, pass@1)
31%
It has input modalities other than text, audio, and video (MIDI/code count as text)
29%
It will score >= 85% on MMLU Pro
27%
It will score >=80% on MMMU

#Google Gemini
Get Ṁ1,000 play money
3 Comments
Sort by:
bought Ṁ30 It will score >70% o... NO

shall be resolved

huh, not sure how the benchmark questions should resolve now that 2.0 Flash is out. Maybe wait for info on 2.0 pro? Also relevant that it is called "2.0 Flash Experimental" for now

https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/#gemini-2-0-flash

Current leading benchmarks, excluding o1 (o1-preview gets 73% on GPQA)

Related questions