Don’t we already have this in https://arxiv.org/abs/2412.06769 ?

Training Large Language Models to Reason in a Continuous Latent Space

Abstract page for arXiv paper 2412.06769: Training Large Language Models to Reason in a Continuous Latent Space

@eapache The idea has been explored, but that's not what I had in mind as a "major AI model".

I think I'll change the title to "state-of-the-art" to make it a little clearer - I'm thinking of something that's used by millions of people, not just a research project.

By when?

I'll use the market close date for this - end of 2030.

I'm confused by this - doesn't the presence of similar features regardless of the "language" the model is using essentially imply that 1. if we have good interpretability, neuralese won't be hard to decipher, and 2., they won't need to be specifically trained for this/it won't confer much benefit?

Good question. Scott suggests that neuralese could be "good for capabilities (neuralese is faster and richer than English) but dooms alignment." But I'm not sure whether that's true, or how confident he is about it.

@TimothyJohnson5c16 The development or prevention of neuralese is a pivotal act determining whether alignment succeeds in the story. The story implies there's a lot of pressure pushing this to happen.