Zubnet AILearnWiki › Sparse Autoencoder
Models

Sparse Autoencoder

SAE
A neural network trained to reconstruct a model's internal activations through a bottleneck with a sparsity constraint — only a few features can be active at once. The learned features often correspond to interpretable concepts (specific topics, linguistic patterns, reasoning strategies), making SAEs the primary tool for disentangling the superposed features inside large language models.

Why it matters

Sparse autoencoders are the microscope of mechanistic interpretability. LLMs pack thousands of features into each layer through superposition, making individual neurons uninterpretable. SAEs decompose these superposed representations into individual, interpretable features. Anthropic used SAEs to identify millions of features in Claude, including features for deception, specific concepts, and safety-relevant behaviors.

Deep Dive

Architecture: the SAE takes a model's activation vector (dimension d_model, e.g., 4096) and encodes it into a much larger sparse representation (e.g., 64K features, of which only ~100 are active for any given input). It then decodes back to d_model and is trained to minimize reconstruction error. The sparsity constraint (L1 penalty on the hidden layer) forces the SAE to use only a few features per input, ensuring each feature is specific rather than diffuse.

What SAEs Find

When trained on LLM activations, SAEs discover interpretable features: a "Golden Gate Bridge" feature that activates on text about the bridge, a "Python code" feature, a "French language" feature, a "sycophantic agreement" feature, and so on. These features are more interpretable than individual neurons because the sparsity constraint separates overlapping concepts that neurons represent in superposition. Anthropic's research found features ranging from concrete (specific entities) to abstract (deception, uncertainty).

Applications Beyond Interpretation

SAE features can be used for more than understanding: clamping a feature to zero suppresses the corresponding behavior (deactivating a "deception" feature), while amplifying a feature strengthens it. This opens the possibility of fine-grained behavioral control without retraining. However, the technique is still experimental — feature interactions are complex, and modifying one feature can have unintended effects on others due to residual superposition.

Related Concepts

← All Terms
← Sparse Attention Speculative Decoding →