Will the next major LLM by OpenAI use a new tokenizer?
43
Ṁ12232025
76%
chance
1D
1W
1M
ALL
The GPT-2 model used r50k_base: vocab size = 50k
The GPT-3 model used r50k_base: vocab size = 50k
The GPT-3.5 model used cl100k_base: vocab size = 100k
The GPT-4 model used cl100k_base: vocab size = 100k
Get Ṁ1,000 play money
Sort by:
@firstuserhere So YES if there's a GPT-4.5/5 that uses a tokeniser not on this list, and NO if there's a GPT-4.5/5 that uses a tokeniser that is on this list?
Related questions
Related questions
Will OpenAI reveal a textless LLM before 2025?
20% chance
What is the next OpenAI LLM logo color?
Will OpenAI release a tokenizer with more than 210000 tokens before 2026?
24% chance
Will there be a OpenAI LLM known as GPT-4.5? by 2033
35% chance
Will the most interesting AI in 2027 be a LLM?
41% chance
Will OpenAI release a tokenizer with vocab size > 150k by end of 2024?
42% chance
What will be true of OpenAI's best LLM by EOY 2025?
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
39% chance
Will OpenAI have the best LLM in 2024?
59% chance
Will Google have a better LLM than OpenAI by 2025?
35% chance