The central operation
Each query produces its own distribution over keys. That distribution mixes values into a new representation for the querying position.
Attention is dynamic information routing.Interactive transformer explainer
See one token decide what matters
Select a query token, switch attention heads, change softmax temperature, and apply a causal mask. The weighted 3D routes are computed from deterministic teaching projections, not extracted from a production model.
Q: current queryK: comparison addressesV: routed content
Attention head
Mask
Attention routes information for one position at a time
A transformer does not assign a single global importance score to every word. At each layer and attention head, every token position creates a query. That query is compared with keys from candidate positions, converted into normalized weights, and used to mix their value vectors.
The distinction between query, key, and value is operational. A query describes what the current position is looking for under one learned projection. A key describes how another position can be addressed. A value carries the information that will be routed if that address receives weight. They are separate linear projections of hidden states, not three copies with human-readable labels.
In the sandbox, select the token it in the package example. The reference head gives the package token a strong structural match. The syntax head prefers nearby grammatical context. The recency head decays with distance. Aggregate averages the normalized distributions to illustrate how multiple heads can contribute different routing patterns.
These are deterministic teaching projections with explicit biases, not a trained language model. Real heads learn distributed features, may combine several behaviors, and change across layers and prompts. The purpose of the lab is to make the mechanics inspectable without pretending that a colorful route is a faithful interpretation of a deployed model.
Scores are relative
Softmax converts a row of logits into weights that sum to one. Raising one candidate's logit reduces the share available to others.
Heads use projections
Different learned Q, K, and V matrices let heads attend through different representational subspaces.
Masks remove routes
A causal mask gives future positions effectively negative-infinite logits before softmax, preventing forward information leakage.
Outputs rejoin the stream
Head outputs are concatenated, projected, and combined with the residual stream before later transformations.
Scaled dot-product attention, step by step
Project hidden states
The layer input at every position is multiplied by learned matrices to create query, key, and value vectors for each head. The same token can therefore expose different features to different heads.
Compute query-key logits
For one query position, dot products compare its query vector with every allowed key. Larger positive values indicate stronger alignment in that head's learned coordinate system.
Scale and mask
Dividing by the square root of key dimensionality keeps logits from growing too large as dimensions increase. Masks then remove illegal routes before normalization.
Normalize with softmax
Exponentiation and row normalization produce nonnegative weights that sum to one. Softmax is sensitive to relative logit differences, which is why scale and numerical stability matter.
Mix value vectors
The output is a weighted sum of value vectors. High attention to a position routes more of its projected value content into the query position's new representation.
Why multiple heads can help
Reference routing
A head can learn features useful for linking a pronoun or dependent phrase to another position. The behavior is contextual and need not align perfectly with a linguistic relation.
it -> package
weight rises when projected features align
weight rises when projected features align
Syntactic routing
Some heads exhibit positional or grammatical patterns, such as attending to a preceding token, separator, or dependency-related location. No single head has to encode a whole parse tree.
verified -> is
local structure can dominate one head
local structure can dominate one head
Recency routing
Distance-sensitive behavior can prioritize nearby context. Relative-position mechanisms, rotary embeddings, masks, and learned content all influence practical routing.
query -> nearby keys
distance changes the teaching bias
distance changes the teaching bias
Distributed routing
Useful computation usually spans heads and layers. A later head can read features written by earlier components, so inspecting one matrix is only a slice of the circuit.
heads -> concatenate -> project
residual stream carries the result
residual stream carries the result
Where attention sits in transformer data flow
Residual input
The current representation at each token position carries embeddings plus changes written by earlier layers.
Shared streamNormalize and project
Layer normalization and learned matrices create head-specific queries, keys, and values.
Head subspacesRoute values
Masked scaled dot products and softmax determine how every query mixes candidate values.
Dynamic weightsCombine heads
Head outputs are concatenated and projected back into the model dimension.
Output projectionResidual update
The attention result is added to the stream so later attention and feed-forward blocks can use it.
Layer stateRESIDUAL STREAMQUERYKEYVALUEMASKSCALESOFTMAXWEIGHTED SUMOUTPUT PROJECTIONRESIDUAL STREAMQUERYKEYVALUEMASKSCALESOFTMAXWEIGHTED SUMOUTPUT PROJECTION
What an attention map does not prove
Weight is not causal importance
A large attention weight shows routing under one head and layer. It does not by itself prove that changing that source token would change the final prediction proportionally.
Values carry transformed content
Two positions can receive similar weights while contributing very different value directions. Looking only at weights ignores what is actually routed.
Later layers can overwrite
The residual stream accumulates many attention and feed-forward updates. A pattern visible early can be amplified, transformed, or canceled later.
Visualization choices distort
Head averaging, token aggregation, thresholding, and color scales can hide uncertainty or invent apparent structure. Preserve the underlying numbers and method.
Why attention mechanics matter for personal agents
A personal agent may receive instructions, prior messages, retrieved documents, tool schemas, and browser observations in one context. Attention is one mechanism that lets each generated position route information from those inputs, but it does not guarantee that the right evidence is present, authorized, fresh, or followed.
For a text-message AI assistant, pronouns and terse follow-ups depend on context. A message such as "send it after it is verified" contains multiple references. The application should preserve structured task identity and verification state instead of hoping attention resolves every ambiguity from prose.
A computer-use cache can place relevant prior observations into context, but retrieval quality determines which keys and values are even available. Attention cannot route evidence that was omitted, and it can still emphasize stale evidence if metadata and freshness are not enforced.
When an AI website-building agent reads files, deployment logs, screenshots, and requirements, clear serialization and section boundaries make relationships easier to represent. Exact paths, current errors, and explicit postconditions reduce reliance on long-range inference.
Super combines messaging and computer-use workflows where model context must stay connected to structured permissions, tools, receipts, and real-world verification beyond the attention computation itself.
Distinguish attention weights from causal feature importance.
Inspect queries, keys, values, masks, and output directions together.
Preserve the exact tokenization and layer/head coordinates.
Use stable softmax and the architecture's defined scaling.
Confirm causal and padding masks before comparing maps.
Test behavior with interventions and downstream metrics.
Keep retrieval, freshness, and permissions outside model intuition.
Label synthetic teaching projections honestly.
Frequently asked questions
Are the displayed weights from a trained transformer?
No. The sandbox computes real softmax distributions from deterministic teaching projections and visible structural biases. It never claims to reproduce a particular model, layer, or learned head.
Why divide by the square root of key dimension?
Without scaling, dot-product variance grows with dimensionality, which can push softmax toward saturated distributions and small gradients. The transformer scales logits by the square root of key dimension.
What does temperature change here?
The teaching control divides logits by an extra temperature. Lower temperature sharpens the largest differences; higher temperature flattens the distribution. Standard attention uses its architecture's defined scale rather than an arbitrary inference temperature knob.
Does a causal mask make attention one-directional?
For a decoder self-attention layer, it prevents a position from reading later token positions. Earlier positions can still influence later ones, and other architectures use bidirectional or cross-attention masks.
Do attention heads have fixed human-readable jobs?
Not reliably. Some heads show recurring patterns, but behaviors can be distributed, polysemantic, prompt-dependent, redundant, or only meaningful when composed with other components.
How does KV cache relate to this diagram?
During autoregressive generation, past key and value vectors can be cached so each new query reuses them instead of recomputing earlier projections. Cache memory grows with layers, sequence length, heads, and key/value dimensions.
Primary research and technical references
Attention Is All You NeedVaswani and colleagues introduce scaled dot-product attention and the transformer architecture.
BERTDevlin and colleagues use bidirectional transformer encoders and masked-language-model pretraining.
FlashAttentionDao and colleagues present an exact, IO-aware attention algorithm that reduces memory traffic.
Attention is not ExplanationJain and Wallace examine limits of treating attention distributions as faithful explanations.
A Mathematical Framework for Transformer CircuitsA mechanistic framework for composing attention heads and residual-stream computations.
Three.js DocumentationThe WebGL library used for the live query-key-value routing scene.
Trace the route, then test the behavior.
Attention becomes useful to understand when queries, keys, values, masks, weights, and downstream effects remain distinct and inspectable.
Explore Super