It's 2025 and most content is still written for humans instead of LLMs. 99.9% of attention is about to be LLM attention, not human attention.

E.g. 99% of libraries still have docs that basically render to some pretty .html static pages assuming a human will click through them. In 2025 the docs should be a single your_project.md text file that is intended to go into the context window of an LLM.

Repeat for everything.

LLMs are natural language computers - trained by natural language, programmable by natural language.

The best way to program LLMs may simply be to communicate clearly and precisely.

UI is pre-AI.

Prediction: LLMs in their current form may not be able to do everything, but AI now has enough momentum that this won't matter. Beam engines couldn't do everything either, but they were enough to set off the Industrial Revolution.

The vast majority of human knowledge is not expressed in text... LLMs do not have that, because they don't have access to it. And so they can make really stupid mistakes. That’s where hallucinations come from.

Don't think of LLMs as entities but as simulators. For example, when exploring a topic, don't ask:

"What do you think about xyz"?

There is no "you". Next time try:

"What would be a good group of people to explore xyz? What would they say?"

The LLM can channel/simulate many perspectives but it hasn't "thought about" xyz for a while and over time and formed its own opinions in the way we're used to. If you force it via the use of "you", it will give you something by adopting a personality embedding vector implied by the statistics of its finetuning data and then simulate that. It's fine to do, but there is a lot less mystique to it than I find people naively attribute to "asking an AI".

[Large language models] require enormous amounts of data to reach a level of intelligence that is not that great in the end. And they can't really reason. They can't plan anything other than things they’ve been trained on. So they're not a road towards what people call “AGI.”

The biggest near-term question in AI:

Will open-source LLMs with decentralized training be competitive with closed-source and centralized LLMs?

AI is eating UI.

The reason AI coding works so well is that the source code of the median app was already slop before LLMs.

People have too inflated sense of what it means to "ask an AI" about something. The AI are language models trained basically by imitation on data from human labelers. Instead of the mysticism of "asking an AI", think of it more as "asking the average data labeler" on the internet.

Few caveats apply because e.g. in many domains (e.g. code, math, creative writing) the companies hire skilled data labelers (so think of it as asking them instead), and this is not 100% true when reinforcement learning is involved, though I have an earlier rant on how RLHF is just barely RL, and "actual RL" is still too early and/or constrained to domains that offer easy reward functions (math etc.).

But roughly speaking (and today), you're not asking some magical AI. You're asking a human data labeler. Whose average essence was lossily distilled into statistical token tumblers that are LLMs. This can still be super useful ofc ourse. Post triggered by someone suggesting we ask an AI how to run the government etc. TLDR you're not asking an AI, you're asking some mashup spirit of its average data labeler.

Half of these AI alignment problems are just human alignment problems. And that's what's also so scary about the language that they use. It's not the machines you want to align; it's me.

Prediction: Established companies will use cherry-picked examples violating broad definitions of AI safety as a way to suppress competition from startups. New technology is always somewhat unreliable, so startups will always be vulnerable to this type of attack.

the real risk with AGI isn’t malice but competence. A superintelligent AI will be extremely good at accomplishing its goals, and if those goals aren’t aligned with ours, we’re in trouble. As I mentioned in chapter 1, people don’t think twice about flooding anthills to build hydroelectric dams, so let’s not place humanity in the position of those ants.