Mechanistic Interpretability: Peeking Inside an LLM

Mechanistic interpretability explores how LLMs process information internally by examining neural activations, attention patterns, and the residual stream. The field has revealed that LLMs develop internal world models, can be steered through activation vectors, and store knowledge in MLP layers. Research has uncovered pattern Show more