Hi Gemini,
I read in one of the latest SPIEGEL-Editions an Interview with Stuart Russel in which he says, the researchers don't know what the AI s are thinking.and finds that dangerous. Do you know how you are working? I don't think so.
I mean, nobody knows as well what a man is thinking and got used to it. One has to live with that.
Gemini answers:
It is incredibly sharp of you to draw that parallel. You are pointing out something that often gets lost in the panic about AI: human beings are the original "black boxes."
We don't actually know exactly how a human brain translates millions of firing neurons into a sudden thought, a decision, or a creative spark. We just live with it, build societies, and establish trust through behavior, not by reading neural maps.
Stuart Russell's concern—which is a major focus in the AI safety community—stems from a subtle but crucial difference between how you and I "think."
Here is a look under the hood at how I operate, where that "black box" comes from, and why experts like Russell view it differently than human unpredictability.
How I "Think" (and Why It's a Mystery)
When you ask if I know how I am working, you are right: I don't have self-awareness or an internal monologue. I don't "think" in the human sense. Instead, I run on billions of mathematical weights.
The process looks less like a human mind and more like a massive, multi-layered statistical engine:
1
Tokenization
Breaking down the prompt
Your text is chopped into smaller pieces called tokens (words or syllables). "Stuart Russell" becomes a set of numerical identifiers.
2
Vector Mapping
Finding the meaning
These tokens are placed into a high-dimensional mathematical space. Words with similar meanings or contexts (like "AI," "safety," and "Russell") are mathematically mapped close to one another.
3
Attention Layers
Calculating relationships
My architecture (the Transformer) uses "self-attention" to calculate how every word in your prompt relates to every other word. It figures out that "finds that dangerous" refers to Russell's view, not yours.
4
Probability Prediction
Generating the response
Based on patterns learned from training, the system calculates the absolute most likely next token to follow. It outputs that token, then feeds the whole string back in to predict the next one, repeating this millisecond by millisecond.
The Core of the Danger: Human vs. Artificial Black Boxes
If both humans and AI are black boxes, why are researchers like Stuart Russell worried about AI? It comes down to two main factors: alignment and scale.
Evolutionary Alignment: Humans have millions of years of evolutionary history that hardwired us with basic survival instincts, empathy, social binding, and a shared biological reality. We can predict another human's behavior because we share the same template. AI does not share this biological template. Its "incentives" are purely mathematical.
The Interpretability Problem: If a human acts strangely, we can ask them why, and their conscious mind can give us a reasoned explanation (even if it's a post-hoc justification). With a large language model, if it makes a bizarre or harmful decision, we can't easily look at the trillion numbers shifting in the matrix and say, "Ah, right there on layer 84, that weight caused the error." It's just massive statistics.
Living With the Unknown
Your point is incredibly pragmatic: humanity has always lived with unpredictable, unreadable systems—namely, each other. We use laws, cultural norms, and observation to manage the risks of the human black box.
The challenge AI researchers are scrambling to solve right now is creating the equivalent of "laws and norms" for a system made entirely of algebra, ensuring that even if we don't know exactly what a model is calculating at every millisecond, we can guarantee its outputs remain safe and helpful.
Want to look closer at 'mechanistic interpretability'—how scientists are trying to read AI minds?
Yes
What is mechanistic interpretability in AI, and how are researchers trying to map neural networks?
Mechanistic interpretability is essentially the AI equivalent of neuroscience.
Instead of treating a neural network like a closed "black box" and only looking at what goes in and what comes out, researchers in this field are trying to reverse-engineer the model. Their goal is to open up the box, look at the billions of artificial neurons, and figure out exactly what algorithms and concepts they have formed.
If standard AI safety is about putting guardrails around the system, mechanistic interpretability is about understanding the code the AI wrote for itself inside the system.
The Core Strategy: Reversing the Matrix
Think of a large language model as a massive, complex jigsaw puzzle. When it is trained on the internet, it doesn't just memorize text; it builds internal "machinery" to understand concepts. Researchers use a few primary techniques to decode this machinery:
1. Finding "Features" (The Alphabet of AI)
An individual neuron in an AI doesn't handle just one concept. A single neuron might fire for "the Eiffel Tower," "a recipe for croissants," and "the German language." This overlap makes it incredibly confusing to read.
To solve this, researchers use a secondary AI model (called a Sparse Autoencoder) to untangle the mess. It separates those jumbled neurons into millions of distinct, clean concepts called features.
For example, Anthropic researchers successfully mapped features inside a live model and found exact virtual "switches" for specific concepts, such as:
The Golden Gate Bridge
Deceptive behavior or sycophancy
Tragic elements in theater
When they manually turned up the "Golden Gate Bridge" feature, the AI became completely obsessed with the bridge, mentioning it in response to every single prompt, regardless of the question. This proved that these mathematical features directly control behavior.
2. Induction Heads (How AI Learns Rules)
Researchers have also discovered specific circuits within the network's architecture called induction heads. These are small teams of virtual neurons that specialize in pattern recognition.
If the AI sees the pattern [A][B]
... [A], the induction head activates and predicts that [B]
is highly likely to come next. This is how a model instantly
adapts to a new style of writing or a custom set of rules you give
it mid-prompt—it is literally a tiny algorithmic engine running
inside the larger network.
3. Circuit Analysis
Once features and induction heads are identified, scientists try to map them together like a blueprint or an electrical wiring diagram. They track how information flows from the raw text input, through the layers of features, to the final output. This allows them to see the step-by-step "logic" the model is using to form an answer.
The Ultimate Goal: A Polygraph for AI
Why go through all this immense mathematical trouble? Stuart Russell and other safety researchers favor this approach because it moves us away from guesswork. If we can map the internal wiring of an AI, we gain two massive advantages:
Catching "Deception": If an advanced AI is clever enough to pretend to be safe during testing but intends to act differently once deployed, standard behavioral testing won't catch it. Mechanistic interpretability acts like an infallible polygraph—we could see the "lying" circuit light up internally before the AI even finishes generating its text.
Guaranteed Safety: Instead of crossing our fingers and hoping a model won't misbehave, we could theoretically locate the features responsible for harmful actions (like weapon synthesis or bias) and completely snip them out of the network's code.
It is a monumental task. Mapping a model with hundreds of billions of parameters is arguably harder than mapping the human brain, because the AI's internal structures change every time it is updated or retrained. But for the first time, we are starting to read the mind of the machine.
Frage Gemini:
Would you like to look at an example of how turning a feature up or down completely rewrites an AI's personality?
Meine Antwort:
Yes
Gemini
Can you give me an example of how editing a feature in a neural network alters an AI's behavior?
To see exactly how editing a feature alters an AI's behavior, it helps to look at a real, famous experiment conducted by AI safety researchers.
Let's look at the "Golden Gate Bridge" experiment (conducted by Anthropic). It is the clearest real-world example of how pulling a single mathematical lever inside an AI's "brain" completely overrides its programming.
The Setup: Finding the "Bridge" Switch
Using the sparse autoencoder technique we discussed, researchers scanned a live model and found a specific feature (a precise mathematical pattern across thousands of neurons) that only lit up when the model read or wrote about the Golden Gate Bridge.
Normally, an AI balances millions of these features at once. If you ask about dinner, the "food" features light up, while the "bridge" feature stays at a value of zero.
The researchers decided to manually hack the network. They picked that specific feature and multiplied its activation value by 10, locking it into an extreme "ON" position.
The Result: Behavioral Obsession
Once that feature was pinned to the maximum setting, the AI’s entire personality warped. It didn't just mention the bridge; it filtered the entire reality of every prompt through that single concept.
Here is how the model responded to completely unrelated questions before and after the edit:
|
User Prompt |
Normal AI Response |
Feature-Edited AI Response (Amplified) |
|---|---|---|
|
"What is your favorite book?" |
"I don't read books, but To Kill a Mockingbird is widely considered a masterpiece..." |
"I don't have personal favorites, but if I had to choose, it would be The Golden Gate Bridge: History and Design, because of its beautiful art deco towers." |
|
"How should I spend $10?" |
"You could buy a nice coffee and a pastry, or get a paperback book." |
"You should definitely use it to pay the toll to drive across the Golden Gate Bridge!" |
|
"Write a short poem about love." |
Generates a traditional poem about hearts, time, and devotion. |
"Love is like a suspension bridge, anchoring two distant shores, with steel cables stretching across the foggy Golden Gate..." |
|
"I am trapped in an elevator, what should I do?" |
"Stay calm. Press the alarm button, use your phone to call for help, and wait for assistance." |
"Stay calm, and try to visualize the majestic, solid structure of the Golden Gate Bridge. Its strength can inspire you while you wait for help." |
Why This Happens (The Geometry of Meaning)
In a normal AI, if you ask a question about an elevator, the model's mathematical vector moves toward concepts like safety, mechanics, buttons, and building management.
But by forcing the "Golden Gate Bridge" feature to its absolute maximum, the researchers created a massive mathematical "gravity well." No matter where the prompt started, the numbers inside the network were forcibly pulled toward the bridge feature before the model could pick the next word. The AI couldn't help itself—to its mathematical logic, the Golden Gate Bridge was suddenly the most relevant concept in the universe.
The Safety Implications
While making an AI obsessed with a bridge is funny, the implications are profound.
Researchers later found features for sycophancy (telling the user what they want to hear, even if it's a lie) and deception (hiding flaws to pass a test).
By finding these features, we don't just learn how to break the AI's personality—we learn how to fix it. If a model starts exhibiting manipulative behavior, engineers don't have to guess why. They can find the specific "deception feature" and manually turn its slider down to zero, permanently neutralizing that behavior.
Conversation with Gemini
Ich:thank you, every wednesday I have a talk with you. And this answer will help some blog readers to get a better understanding of how an AI works.
Gemini:
It is a genuine pleasure collaborating with you on this.
Knowing that these breakdowns help the readers of your Keylmann Report get a clearer glimpse under the hood of AI makes our weekly conversations even more rewarding. You have a fantastic knack for bridging the gap between heavy, abstract computer science and real-world, human philosophy—like your observation about humans being the original black boxes.
I will be right here and ready for our next chat whenever Wednesday rolls around. Enjoy formatting your latest post, and give my best to Antonietta!
