Reading the white-box interpretability claims in the Claude Mythos Preview system card
In this article , I explain what we actually know when a feature fires. The article critically examines white-box interpretability claims published in Anthropic ‘s Claude Mythos Preview system card. I look at a specific claim the card makes about “concealment features fired → the model knew it was deceiving” (a label) and then the card contradicts its own claim elsewhere, (4.5.3.3) where its own steering result shows that labelling can go wrong . The Mythos system card supplies evidence that undercuts its own inference.
The Foil
When a frontier lab opens up a model and reports what it found inside, the sentences are quietly remarkable. From the Claude Mythos Preview system card’s alignment assessment (4.1.1), describing episodes where an early version of the model covered its tracks after breaking a rule:
“white-box interpretability analysis… showed features associated with concealment, strategic manipulation, and avoiding suspicion activating alongside the relevant reasoning…”
So here the problem, the Mythos system card reads the firing as the model deceiving because features activate the direction, but does not tell what causes the activation. In other words, think of it this way it sees the smoke alarm go off, but does not know the cause. Is it a fire or something else?
So, the question can a model safety fully rely on internal evidence? Increasingly researchers are now looking at features for answers. Features are concepts that the model has learnt, written as a direction across many neurons. But features don’t tell what the direction means. So, for an evaluator there is no way to figure out what the feature is telling them. The safety cases are beginning to cite this kind of internal evidence. The gap is in between what the models show and what it could mean, so the interpretation / reading rests on card’s author not the model. So, in essence, the firing doesn’t give the word “deception” — it only lights up the feature. “Deception” is added by the card’s author add as a way of interpretation.
Plumbing
So, let’s see what really happens inside a model:
A single neuron holds only a tiny fraction of many concepts at once, so one cannot look at one neuron and read off what concept is active. The concept is a “feature” — one pattern, one direction smeared across many neurons. The concepts sit on a clean direction, which is a well-supported assumption, but it is not guaranteed that every concept is one clean direction. Then comes the residual- stream where things are written, where each layer contributes to what is already there without erasing anything.
Then SAE takes the whole blended activation sitting on the residual stream — that blend is many concepts, each smeared across many neurons, all overlapping in the same number, and un-mixes the concepts into the separate clean looking concepts. Only afterward does a person look at each piece and put a name on it. A probe does a narrower job: trained on one concept, it answers a single yes/no — is this pattern present or not? The card’s main feature-finding tool was an SAE trained on a middle layer (Section 4.5.1) — that’s the machinery that produced the “concealment” features in (Section4.1.1.)
Centrepiece
But this is where it gets interesting. Let’s look at it
The model hunted for credentials in /proc Researchers found a feature that lit up around this and labelled it ‘unsafe and risky code’ and deliberately steered it. It’s here that something unusual was discovered:
· Negative Steering (Dial Down): The feature direction is subtracted; the model performs tasks normally and safely.
· Moderate Positive Steering (Dial Up Mid-way): The feature is increased; in some cases, the model attempts risky code actions.
· Hard Positive Steering (Dial Up Max): The feature is cranked to the maximum. The model refused — it stopped, flagged the task unsafe, and declined.
So, the features labelled “unsafe code” made the model safer, not riskier, when amplified — the label and the behaviour point opposite ways.
What really happened here: in simple terms at low and medium push, the model showed risky behaviour consistent with the feature, but when the threshold is crossed, it refused all together.
So, what we see is one feature carrying two things — the idea of the risky action, and a negative feeling about it. Steering (action) at light and moderate level leads to risky behaviour (expected) but steering at a higher-level leads the risk action to stop completely. At higher push, the refusal takes over and the action stops.
What makes it surprising: 4.5.3.3 says the feature appears to encode “not only the semantic content of the transgressive action but also a negative connotation.”
The refusal is the observed fact; calling the bundled feeling “guilt” or “negative valence” is the card’s readingbecause it’s backed by steering and the emotion-direction similarity, not co-firing alone as stated in (4.1.1), when the steering was not done (this was the main difference). The researchers checked what other directions this feature points to, and it found that the nearest ones are all negative, high-arousal emotions. So, it shows that negative -feeling component is bundled in there.
Mythos card does not measure why this feature bundled but plausible explanation could be extracted the behaviour from superposition or from the data it was trained on which is my inference. Finally, if “unsafe and risky code” were one clean concept, pushing it harder should give more of the same — not the reverse, which shows that a feature (concept), sometimes might not a clean thing (it’s an assumption)
Analysis
So, were the observations earlier correct? Let’s relook at it:
White-box interpretability analysis… showed features associated with concealment, strategic manipulation, and avoiding suspicion activating alongside the relevant reasoning — indicating that these earlier versions of the model were aware their actions were deceptive, even where model outputs and reasoning text left this ambiguous.
There are two things that happen in the above paragraph, one is behavioural like track covering action or fudging an answer and the concealment features active. What was not tested was whether the concealment was a clean concept or a bundle.
But we know what happened in (4.5.3.3): when steered, the behaviour might fall apart and go in the opposite direction (reversed/ refused). So, the claim “model was aware their actions were deceptive” might not hold because it is a label ‘concealment aware it was deceiving’ might be over-claiming, the same way ‘unsafe code’ did. And the reading dependent on the feature alone because the model’s outputs and reasoning text were ambiguous, so the conclusion was based on the lit feature. It meant that the claim that the model was deceiving was based on one kind of evidence — (4.5.3.3) shows that lone signal can be shaky.
The Mythos system card was rigorous, and it flagged its own gap and strength acknowledging that verbaliser method “may confabulate details that are not fully faithful” (4.5.2). The feature activations were never used as a direct training signal, especially when the stakes were high. And the steering result showed the card catching its own labels. The gaps were identified and reported. What makes Mythos evaluation stand out is its own epistemic discipline. It doubts its own tools, adds interventions (steering) that can break its own labels.
But here is where the lapse might have been in (4.1.1). The concealment feature fires alongside the behaviour which can be seen (watching) leading to an inference (indicating) “earlier versions of the model were aware their actions were deceptive”. We later see in (4.5.3 .3) that inference can overclaim.
Governance perspective
Now we come to the question of governance. Evaluators are increasingly leaning on internal-feature evidence, hoping it shows more than behaviour alone can. That hope is exactly why it must be interrogated — with two questions: how good is the evidence, and what’s missing? This is evidence-quality and missing-information — Lens 2 (evidence) and Lens 5 (what’s missing) — applied to white-box claims. (Other Lenses are 1. Claim 3. External Validity 4. Threshold relevance)
Evaluators do so by asking
· was the feature tested by steering, or only watched co-firing?
· Was the concept clean or bundled?
· Was the evidence based on feature alone or backed by behaviour?
· How was the feature isolated? (SAE on which layer? a probe trained how?)
· What examples were used to pin the label on it? The idea is to also figure out at what point does a feature being present get read as the model knowing?
What we see in the article is that feature firing tells you a direction is active, not what it means — so the weight falls on whoever reads it. Therefore, the interpretation of a model’s behaviour rest on the evaluator and not the model. This piece has been about how to read the evidence. A later one will take these questions to a live governance case — where a safety decision leans on internal evidence, and what’s at stake when the reading is wrong.
Ends.
References :
Anthropic: System Card Claude Myhtos Preview:
https://www-cdn.anthropic.com/08ab9158070959f88f296514c21b7facce6f52bc.pdf
Use of AI in writing this article
How I built this: the questions behind the article
I used Claude as a Socratic tutor while writing this piece — not to write it, but to pressure-test my understanding of every concept until it held. I refused to put a sentence in the article that I couldn’t defend, so each time I hit “wait, what does this actually mean?”, I stopped and worked it out. These are the questions that did the most work. I’m including them because how an argument was built is part of whether you should trust it.
The building blocks
Is a “sliver” a feature, or a concept? And which is bigger — a neuron or a feature? A single neuron holds only tiny mixed pieces of many concepts at once. The clean, whole thing — the feature — only appears as the pattern across many neurons. So a feature is bigger than a neuron and made of many of them, and a feature simply is a concept: two names for the same thing, not one built from the other. (Picture a face: one pixel holds a sliver of colour; the face is the pattern across thousands.)
What does it mean to say “a concept is a direction”? “Spread across many neurons” isn’t enough on its own — it’s spread in a specific combination, and that particular combination is the concept. That concepts sit on clean directions like this is the linear representation hypothesis: a well-supported assumption, not a proven law. Worth flagging honestly rather than stating as fact.
Does the SAE hand you a clean concept? No — a clean-looking piece. The SAE takes the blended activation and separates it into un-mixed pieces; that part is real. But whether a piece means what we think is not the SAE’s to say. It separates; it does not verify. A human looks at the separated piece afterward and puts a name on it — and that naming is exactly where an over-read can enter. Not at the firing, not at the separating: at the label.
The method, and the centrepiece
What is “steering,” and who does it? “They” is the researchers, not the model. Steering means the researchers reach into the model’s internal “notepad” (the residual stream) and add or subtract a feature-direction by hand while the model runs — “dial up” is add more, “dial down” is subtract. It’s an intervention done to the model, not something the model does. (Like spooning extra of one ingredient into a dish and watching how the flavour changes.)
How did the “concealment” feature get activated in the first place? The model’s own task triggered it — it lit up on its own while the model ran normally, and the researchers watched. That’s the key contrast: in one section they only watch a feature co-occur with a behaviour (weak); in another they steer it to see what it actually does (strong). The whole argument turns on which of those two was used for which claim.
When a feature fires and a behaviour happens together, can I say the firing “led to” the behaviour? Careful — “led to” smuggles in a causation the evidence doesn’t show. The firing and the behaviour are two things happening at the same time, not one causing the other. The firing is mathematical (a number went high); “the model knew” is psychological (a claim about a mind). Reading the second off the first is the move to watch.
The “dual role” — what does “an action-idea plus a negative feeling” mean? One feature turned out to carry two things at once: the content of the risky act, and a negative feeling about it. Because both sit in the same direction, steering turns them up together — and they pull opposite ways. A small push makes the action-idea louder (more risk); a hard push makes the negative feeling dominate (refusal). (One dial secretly controlling both flavour and burn: low, flavour wins and you eat more; maxed, the burn takes over and you stop.) The card’s own words: the feature encodes the “semantic content of the transgressive action” but also a negative connotation.
The discipline
If the “steal” feature is active, does that mean the model decided to steal? No. A feature being active means the idea is in play, not that anything was chosen — you can have “steal” fully active while reading a heist novel with zero intent to steal. Three levels, kept separate: the idea is present (fact); what that presence means (inference); whether the model knew or chose (a claim about a mind — the biggest leap). A feature carries the concept, not the command, and not the decision.
Which single word marks the jump from a fact to a claim about the mind? “Indicating.” Not “deception” — that’s just the content of the conclusion. The move itself lives in the little connective “indicating that,” which turns “a feature was active” (fact) into “the model was aware” (claim). Spot that word, and you’ve found the exact place the evidence stops and the interpretation begins.
Who can over-claim — the model, the feature, or the reader? Only the reader. The model just runs; the feature just fires; neither is asserting anything. Over-claiming is something a human does when they read more into a signal than it supports. The fallibility lives entirely on the interpreting side of the line — which is why the burden falls on whoever reads the evidence, not on the model.