https://amitprakash.in/blog/biocircuit/ Recent content in Biocircuit on Amit Prakash Hugo en-US Copyright © 2025, Amit Prakash. Sat, 19 Sep 2026 12:00:00 +0530 https://amitprakash.in/biocircuit/ Sat, 19 Sep 2026 12:00:00 +0530 https://amitprakash.in/biocircuit/ <p><strong><a href="https://amitprakash.in/">Amit Prakash</a> &amp; <a href="https://x.com/infinitasium">Lalithadithya N</a></strong></p> <h2 id="the-biology-of-a-language-model">The biology of a language model</h2> <p>LLMs are blackboxes where the behaviour of the model is a mystery. Mechanistic interpretability tools bridge the gap by allowing us to see how models reason about a certain prompt within the model. Anthropic’s <em><a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">On the Biology of a Large Language Model</a></em> and circuit-tracer identify interpretable features and map their interactions through attribution graphs. Studies with Claude 3.5 Haiku reveal evidence of intermediate reasoning, advance planning in poetry, and representations shared across languages. However, attribution graphs are approximations of the actual behaviors within the model itself, they replace the underlying MLP activations with a component which is more sparse. They suggest explanations that researchers test by changing the internal activations and observing the effects it causes.</p>