Essay · May 2026
LLMs Are AGI
I will begin this post by putting my 90’s sitcom valedictorian hat on: Merriam-Webster defines general as ‘involving, applying to, or affecting the whole (rather than just part), as well as being common to many, widespread, or not specialized’. In this post is I will claim that, given this definition, we should not be asking ourselves when/if LLMs will reach AGI but rather we should acknowledge the fact that not only have LLMs reached AGI but they embody this property more than any person that’s ever lived. LLMs are the General Intelligence GOATs.
-
General is defined as …. While LLMs are trained to maximize the likelihood of the next token, having been trained on the entire internet. I know this sentence is a common trope at this point, indeed a 2026-trained gpt2 equivalent would likely perfectly complete this sentence, with low perplexity as well, but I find that the immediate consequence from this is nearly never stated - at each point in time, the LLM’s goal is not to showcase superhuman abilities nor is it to generate novel research - it is to simulate what the modal person who would face such a prompt, and respond accordingly.
-
Therefore, LLMs are superhuman in being human, they can predict the future of a sentence much better than any human that ever lived.
-
This does not mean that they cannot give us new insights. Sometimes you don’t know how to get somewhere but you know where you want to go, leverage LLMs as Simulators to narrow down the PDE that you are solving.
-
You can solve in two ways, either specifying initial and end states or a point and a velocity
-
This is exactly what memory does, since there is no free lunch
-
How this connects to societies and ask vs guess culture. The former is Pareto dominant in getting what you want, emotions aside ( the problems with violating common knowledge)
-
An immediate, and indeed a strong rebuttal to my core claim would be “next token prediction is the optimization target at pre training only, once we reach mid/post training we use RL and the model learns how to be helpful (honest, and harmless). However, it has been shown that RL does not commonly give rise to new abilities, that had heretofore been entirely absent from the model. rather it changes the predisposition of the model, alters the distribution if you’d prefer maths-talk over anthropomorphisms, to those that have been shown to maximize the reward. Under our previous framing, we do not alter the fundamental nature of the interaction but rather slightly constrain it. Instead of asking the model to simulate the modal reaction from all its training, we ask it to simulate the modal reaction from all its training with the assumption that the interaction taking place is a dialogue between a “leader” and his helpful, scared to disappoint, politically-correct assistant.
• • While I have come off as a bit of a Debby-downer, and indeed I don’t see current paradigm giving rise to ASI, there is obviously immense power {already one of the most consequential revolutions in history, with potential to be the most, even barring a paradigm shift} to be released and captured. How this apparent contradiction is to be explained will be discussed here:
Outline
First Piece
- Start by discussing AGI, why training for next token prediction given today’s pretraining pipelines and loss functions does not lead to ever growing intelligence but rather ever growing knowledge.
- Differentiate knowledge vs intelligence. Taste. Noticing Discrepancies and gaps in your understanding, its not that you don’t know how to continue, its that you don’t know how a thing you already believe got there.
- Can reward-maximization RL alone cause the development of taste? Steven Byrnes Hypothesis
- The mismanaged Genius Hypothesis
Potentially Second Piece
- Few Shot Examples -> CoT -> Reasoning Models. Analogous to the diffusion process where we take small short steps along what we previously believed makes more sense, but we reconsider once we get there. It allows you to explore the most promising direction in your local neighborhood rather than immediately committing ahead of time.
- Is
Third Piece
- Neuro-Modulators and Hormones as global state, Neurons as local variables. System Prompt vs user messages
Resources
training as Manifold Learning, Inference as manifold traversing
- https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators
- https://substack.com/@deepmanifold/p-194657662
The Prompt Ladder as psychological makeup, sense of self
- https://www.anthropic.com/research/persona-vectors
- https://www.anthropic.com/research/persona-selection-model
SAEs VS prompts - where is the difference?
https://x.com/i/trending/2043478063963422851 https://ericjmichaud.com/quanta/ https://x.com/cwolferesearch/status/1949836549727686995 https://www.lesswrong.com/posts/DiJT4qJivkjrGPFi8/impact-agency-and-taste https://www.lesswrong.com/posts/FNJF3SoNiwceAQ69W/behaviorist-rl-reward-functions-lead-to-scheming https://www.jasonwei.net/blog/asymmetry-of-verification-and-verifiers-law https://infinitefaculty.substack.com/p/memorization-vs-generalization-in https://x.com/carnot_cyclist/status/2041479655035679163 https://builders.ramp.com/post/why-we-built-our-background-agent https://le-wm.github.io/ https://www.lesswrong.com/posts/oxvnREntu82tffkYW/we-need-a-field-of-reward-function-design https://x.com/odysseus0z/status/2030416758138634583 https://www.lesswrong.com/posts/9rCTjbJpZB4KzqhiQ/you-can-t-imitation-learn-how-to-continual-learn