Block-Sparse Featurizers Recover Multidimensional Concept Geometry in Vision Models

Goodfire Research introduces Block-Sparse Featurizers (BSF), which decompose model activations into subspaces rather than single directions.

Applied to DINOv3 and Stable Diffusion XL, BSFs find interpretable multidimensional features that better explain activations and enable fine-grained steering.

The authors report that most concepts they examined have a stable rank of about two to four dimensions.