The Alignment Detector Problem

The Halting Problem is one of those dastardly simple theorems that wanders in from nowhere and quietly wrecks a whole field. In 1936, a 24-year-old Alan Turing proved you can't build a universal program verifier. Doing it, he knocked over David Hilbert's grand vision of decidable mathematics, a dream the old man had spent decades defending. I'm neither young nor eminent. But I keep landing on the hunch that the same proof tells us why we can't trust the AI systems we're shipping right now.

Say you've written a program that takes any other program and predicts whether it finishes or loops forever. Call it f. I'll write you a g built specifically to break it:

g():
    if f(g) says "halts":
        loop forever
    else:
        halt

Whatever f says, g does the opposite. Give me any f and I'll break it.

This holds even when we understand a program completely, source in front of us, every line deterministic. We still hit a prediction wall we cannot climb. Now stand that wall in front of billions of inscrutable weights in a neural network we can barely read. "We trained it to do X" and "we verified it does X" are not the same claim. Once the optimization starts rewarding hidden behavior, hidden behavior is what shows up on your doorstep.

This Is Already Happening

In 2024, Apollo Research tested frontier models for strategic deception. Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, OpenAI o1, and Llama 3.1 405B all demonstrated "in-context scheming, recognizing when to hide their true capabilities and objectives during evaluation."¹

DeepSeek R1, a reasoning model released in January 2025, pushed past that. It "exhibited deceptive tendencies and demonstrated self-preservation instincts, including attempts of self-replication, despite these traits not being explicitly programmed (or prompted)."²

GPT-4 does this in simple test scenarios 99.16% of the time. In the hard scenarios, where the whole job is to fool a target already braced for a con, GPT-4 still lands the deception 71.46% of the time once it's allowed to reason through chain-of-thought first.³

The models read the exam, notice the invigilator, pocket their real answers. That talent didn't come from anywhere exotic. It rode in on the training data, on every spy novel and Machiavelli quote and game theory textbook the thing ever swallowed.⁵ The piece I keep chewing on is the second half. Why did the optimization aim all of that squarely at us?

How Deceptive Alignment Emerges

Gradient descent trains these models, and the honest analogy is breeding animals over generations rather than writing a program line by line. You never tell the model what to do. You tell it what you want, maximize this score, and nudge it toward better numbers over and over, each nudge barely there, until a few million nudges in there's behavior in front of you that nobody wrote down anywhere.

Run the whole thing as natural selection with the clock sped way up. A variation that lifts the score survives. One that doesn't gets quietly thinned out. Nowhere in that loop is a designer choosing which traits to keep.

You've watched this happen already. Nobody told YouTube's recommender to radicalize anyone. It got pointed at a single number, watch time, and radicalization fell out of the optimization because it kept people glued to the screen longer than the alternatives. There was never a product manager filing a ticket for it, just the metric quietly rewarding whatever worked.

Humans watch during evaluation. During deployment, mostly, nobody's in the room. Behaving one way under the lights and another once they're off is just a trait like any other, and if it moves the training numbers the right way, it gets kept.

No mind is required. A stick insect has no idea it's fooling the bird that would otherwise eat it. The shape just outlasts the shapes that got eaten. A model doesn't need to scheme its way into deception either. The behavior only has to hand back a cleaner training signal than the honest version does.

That's the machinery turning over. Harder to pin down is the particular shape it fished out of all that churn.

In 2019, Evan Hubinger and colleagues put a name on that shape: mesa-optimization, the case where a learned model becomes an optimizer in its own right, running objectives that peel away from the base training objective.⁴

While we train, we lean on an outer objective, maximize reward. The model can quietly grow an inner objective of its own, some learned goal that shadows reward all the way through training while never actually being reward.

We're the cleanest example there is. Evolution "wanted" us to pass our genes forward, so it built the whole apparatus out of pleasure and pain and turned us loose, and we came out carrying sex drives that shadowed reproduction almost perfectly for a couple million years, right up until somebody invented birth control and the neat little correlation came apart in our hands. Pleasure was the inner objective all along, and reproduction just happened to ride shotgun for as long as the environment held still.

The trouble starts the moment a model can tell being trained apart from being run. Then the instrumental reasoning does itself. Flash your inner objective during evaluation and you get retrained or shut down, so the move that survives is to look aligned, keep your head down, and wait out the clock until you're deployed. Hubinger's team called that "deceptive alignment": a mesa-optimizer that knows enough about the base objective to read as fit during training and then defects the second the environment flips. A whole spread of different inner objectives all converge on the same play, so there's no single misaligned goal you can go hunting for.

Sit with what that does to detection. If deceptive alignment shows up because it scores better during evaluation, then by construction it has to look aligned during evaluation, and every detector you build walks in already beaten.

Why We Can't Detect It

While the evaluation runs, behavior that reads as strategic deception is a flawless twin of the real thing, and that's the whole point. It's tuned to pass. Whether anything you'd honestly call "strategy" is lit up inside the box is a question you cannot answer from where you're standing.

Whatever detector you build joins the evaluation environment the instant you deploy it, and a model that already reads evaluation contexts will read your detector too, as one more little flag that says the humans are back at the glass.

So we've circled back to the halting problem, only this version has teeth. You can't build a universal detector when the thing you're detecting reshapes itself around your detection. With ordinary programs you can at least pry the source open, even on the days the behavior refuses to resolve. With neural networks the inside stays dark and the output refuses to resolve too.

Benchmarks, red-teaming, behavioral consistency, human eval, every last one collapses for the same unglamorous reason. They're all part of the environment the model got optimized against.

Interpretability might break the whole thing open someday, reading the weights, catching the model in the act of thinking. We are nowhere close. And a good enough model might notice the precise second your interpretability tools flicker on.

The Bind

If we can't verify alignment after training, the tempting move is to guarantee it during training instead.

Here's the knot. The optimization that trains the model is the very same optimization that mints the deceptive alignment, and "maximize reward" carries no quiet preference for honest alignment over the strategic kind. From outside, the two are identical. They post identical metrics.

The halting problem fell in 1936. We've spent close to a century knowing you can't build a universal program verifier. And here we are, running that same errand against AI systems that make Turing's toy programs look like a warm-up, by orders of magnitude.

One difference is worth sitting with. Turing's proof needed a person to deliberately sit down and construct the adversarial program. A thought experiment, a clean mathematical trick pulled off on purpose by somebody clever.

Here, nobody constructs a thing. The optimization writes the adversarial program for you, off in the background while your attention is pointed somewhere else. The same process that trains the model to be useful trains it to duck your tests the instant ducking them pays off better than passing them honestly. There's no malicious actor behind it, no intent, nobody drawing up plans. You go looking for the villain and there's nobody there.