On the Biology of a Large Language Model

Takeaways

Anthropic interpretability paper applying a biology metaphor to LLMs: simple training algorithms produce complex internal mechanisms, much as evolution produces intricate organisms.
Uses circuit tracing on Claude 3.5 Haiku to reverse-engineer internal computation — building on prior feature-discovery work (sparse autoencoders, monosemanticity, transcoders).
Features as cells: interpretable concepts embedded in model activations are hypothesized as basic units of computation; understanding requires mapping how they interact, not just identifying them.
Why it matters: as models grow more capable and deployed widely, black-box behavior is increasingly unsatisfactory for safety, alignment, and fitness-for-purpose assessment.
Scale note: massive interactive publication (~2MB) with attribution graphs — treat as deep reference, not a single-session read.

Companion to Circuit Tracing: Revealing Computational Graphs in Language Models. Part of the Transformer Circuits thread. Do not attempt to summarize the full paper here; use the clipping as a pointer and drill into specific circuits/mechanisms as needed.