Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Workshop on Sparsity in LLMs (SLLM): Deep Dive into Mixture of Experts, Quantization, Hardware, and Inference

Antipodal Pairing and Mechanistic Signals in Dense SAE Latents

Alessandro Stolfo · Ben Wu · Mrinmaya Sachan


Abstract:

Sparse autoencoders (SAEs) are designed to extract interpretable features from language models, yet they often yield frequently activating latents that remain difficult to interpret.It is still an open question whether these \textit{dense} latents are an undesired training artifact or whether they represent fundamentally dense signals in the model's activations.dense latents capture fundamental signals which (1) align with principal directions of variance in the model's residual stream and (2) reconstruct a subspace of the unembedding matrix that was linked by previous work to internal model computation.Furthermore, we show that these latents typically emerge as nearly antipodal pairs that collaboratively reconstruct specific residual stream directions.These findings reveal a mechanistic role for dense latents in language model behavior and suggest avenues for refining SAE training strategies.

Chat is not available.