In 2024, will METR or Google announce the results of a METR eval on a Google LLM?
Mini
7
Ṁ189Jan 1
72%
chance
1D
1W
1M
ALL
METR = formerly ARC Evals (https://metr.org/)
if METR/Google reorgs and has a clear successor org, that org also applies for the purpose of this market
central YES cases:
if Google releases a model card with something like Gemini Supermega 2024 edition with METR exfiltration eval results, like OpenAI did for GPT4 technical report
does not have to be the specific exfiltration eval.
does not have to be included in initial model release paper. does not have to be specifically in a paper.
does not have to be any specific eval granularity. "METR ran the eval and it was all OK" would be ... annoyingly vague from whoever would write it, but it would count.
has to be confirmed-ish by Google and/or METR. can't be just a Twitter rumor.
Get Ṁ1,000 play money
Related questions
Related questions
Will Google announce that it's going to power Google Translate with an LLM-only based system (before 2024 end)?
60% chance
Will Meta AI's MEGABYTE architecture be used in the next-gen LLMs?
45% chance
In what year will Google or Meta release a commercial facial recognition search product?
Will YouTube Comments make it into a major LLM by EOY 2027?
63% chance
Will there be a clear way to integrate LLMs with ads by the end of 2024?
78% chance
Will OpenAI have the best LLM in 2024?
59% chance
Will the new LLM released by Meta be open-source?
61% chance
Will Google have a better LLM than OpenAI by 2025?
35% chance
[Metaculus] Will Google implement a feature to explain targeted Google Ads before 2026?
50% chance
Will Google cancel an LLM-based product by end of 2025?
59% chance