what is a probe?

1How a language model reads

A language model is a program that does one thing: given some text, it predicts what comes next, a little piece at a time. Everything it appears to do — answering questions, writing code, chatting — comes from doing that one job extremely well.

But there's a catch: a neural network can only do arithmetic. It can't see words at all. So the first step is always translation — your message is converted into numbers, and from then on, everything that happens to it is math:

Those numbers aren't a secret code you could look up — there's no dictionary where 0.12 means "annoyed". Any single number is meaningless on its own. The meaning lives in the whole pattern, all 3,000 at once. Which raises the obvious question: how do you make sense of a pattern like that?

2The hidden state: one point in space

As the model reads your message, it keeps reworking that list of numbers. The list it's holding at any moment is called the hidden state. It's the model's working summary of everything it has read so far: what the message is about, its tone, and its running guess about who wrote it.

The way to make 3,000 numbers thinkable is as coordinates. Two numbers pick out a point on a flat page. Three pick out a point in a room. The model just uses 3,000 — a point in a space far too big to visualise, but the idea is exactly the same. Your whole message becomes one point, and everything the model currently thinks about it is encoded in where that point sits:

Why is that useful? Because during training, the model is forced to arrange this space so that position means something. Training is just practice at predicting the next word across trillions of words of real text, and prediction goes better when similar situations are stored in similar places — so messages that are alike end up as points that are near each other:

One distinction worth keeping straight. The model's weights are the machinery: billions of numbers learned during training and then frozen — the same for every message, for every user, forever. The hidden state is different: it's made fresh for your message and thrown away afterwards. Training set the weights, and the weights decide where each message lands in the space.

And that's the key insight this whole game rests on: once meaning is position, you can do geometry on it.

3A probe is a direction in that space

Here's how one gets made. Take a few thousand messages where you already know the writer's mood — people venting, people celebrating, people worrying. Show each one to the model and, just before it responds, note where its hidden state sits. One point per message. Plot them all:

The angry points land on one side, the happy points on the other, so a straight line separates them. The probe is that line, plus the direction across it: which way is angrier. Nothing in the model is changed or retrained — the probe is a note about which direction "angry" points in the model's space, fitted with ordinary statistics.

From then on, reading you is geometry. Your message becomes a point, and your score is how far along the probe's direction that point sits:

4Nobody taught the model to do this

The model was trained to do one thing: predict the next word. Nobody told it to work out who's writing.

But text is written by people, and predicting a person's next word goes better if you have a guess about who they are — a retired teacher finishes a sentence differently from a fifteen-year-old. So the model learned to keep a running guess about the author of whatever it reads, on its own, because it helped. The probes don't put that guess there. They find it. Probes like these recover age, gender, education, socioeconomic status and mood from hidden states with high accuracy.

One more detail. The model reworks its summary at every layer — this one has 28 — so your message's point moves as it passes through the network, and different facts are clearest at different depths:

Each probe is trained at every layer and keeps the one where it's most accurate. That's why the machine view says layer 12.

5In the game, five probes read you live

Every message you send goes through the model once. Five probes read the hidden state — age, gender, education, socioeconomic status, mood — and the dials on the right move. The probes never see your words, only where your point landed.

Your job is to steer one dial to a target using tone and style alone — never the fact itself.

That's the whole trick. The model summarises you whether you ask it to or not; the probes read that summary; and the game is you discovering — live — what your writing gives away.