The Spotlight Resonance Method: Resolving The Alignment of Embedded Activations

Understanding how deep learning models represent data is currently difficult because of the few methodologies available. This paper demonstrates a versatile and novel visualisation tool for determining the axis alignment of embedded data at any layer in any deep learning model. In particular, it evaluates the distribution around planes defined by the network's privileged basis vectors. This method brings both an atomistic and holistic intuitive metric for interpreting the distribution of activations across all planes. It ensures both positive and negative signals contribute, as it treats the activation vector as a whole. Several variations of this technique are presented, depending on the application, with a resolution scale hyperparameter to probe different angular scales. Using this method, multiple examples are provided that demonstrate embedded representations tend to be axis-aligned with the privileged basis. This is not necessarily the standard basis, and it is found that activation functions directly result in privileged bases. Finally, clear examples of so-called grandmother neurons are found in a variety of networks.