Table of Contents

A new way to explain LLM-based AI

Thanks to Eliezer Yudkowsky, I just found my new favorite way to explain LLMs—and why they’re so strange and extraordinary.

Here’s the post that sent me down this path.

And here’s the bit that got me…

well-posed problem = prediction of next token of answer

Like—I knew that. And I have been explaining the power of LLMs similarly for over two years now. But it never occurred to me to explain it in this way. I absolutely love it.

Typically, when you’re trying to explain how LLMs can be so powerful, the narrative you’ll get from most is…

The problem with this argument—which Eliezer points out so beautifully—is that—with an adequate understanding of the world—there’s not much daylight between next token prediction and answer prediction.

So, here’s my new way of responding to the “just token prediction” argument, using 5 levels of jargon removal.

The 5 Levels of LLM Understanding

TIER 1: “LLMs just predict the next token in text.”

TIER 2: “LLMs just predict next tokens.”

TIER 3: “LLMs predict the next part of answers.”

TIER 4: “LLMs provide answers to really hard questions.”

TIER 5: “HOLY CRAP IT KNOWS EVERYTHING.”

That resonates with me, but here’s another way to think about it.

Answers as descriptions of the world

If you understand the world well enough to predict the next token of an answer, that means you have answers. 🤔

Or:

The better an LLM understands reality and can describe that reality in text, the more “predicting the next token” becomes knowing the answer to everything.

But “everything” is a lot, and we’re obviously never going to hit that (see infinity and the limits of math/physics, etc.).

So the question is: What’s a “good enough” model of the universe—for a human context—to be effectively everything?

Human vs. absolute omniscience

If you’re tracking with me, here’s where we’re at—as a deductive argument¹ .

100% agreed.

But the human standard for knowing everything isn’t actually knowing everything. The bar is much lower than that.

The human standard isn’t:

These are—as far as we know of physics—completely impossible to know because of the limits of the physical, atom-based, math-based world. So we can’t ever know “everything”, or really anything close to it.

But take that off the table. It’s impossible, and it’s not what we’re talking about.

What I’m talking about is human things. Things like:

These questions are big. They’re huge. But there’s an “everything” version of answering them (which we’ve already established is impossible), and then there’s the “good enough” version of answering them—at a human level.

I believe LLM-based AI will soon have an adequately deep understanding of the things that matter to humans—such as science, physics, materials, biology, laws, policies, therapy, human psychology, crime rates, survey data, etc.—that we will be able to answer many of our most significant human questions.

Things like:

Those are big questions—and they do require a ton of knowledge and a very complex model of the universe—but I think they’re tractable. They’re nowhere near “everything”, and thus don’t require anywhere near a full model of the universe.

In other words, the bar for practical, human-level “omniscience” may be remarkably low, and I believe LLMs are very much on the path to getting there.

The argument in deductive form

Here’s the deductive form of this argument.

🤯

Guess what? We do it too…

Finally there’s another point that’s worth mentioning here, which is that every scientific indication we have points to humans being word predictors too.

Try this experiment right now, in your head: Think of your favorite 10 restaurants.

…

As you start forming that list in your brain—watch the stuff that starts coming back. Think about the fact that you’re receiving a flash of words, thoughts, images, memories, etc., from the black box of your memory and experience.

Notice that if you did that same exercise two hours from now—or two days from now—the flashes and thoughts you’d have would be different, and you might even come up with a different list, or put the list in a different order.

Meanwhile, the way this works is even less understood than LLMs! At some level, we too, are “just” next tokens predictors, and it doesn’t make us any less interesting or wonderful.

Summary