Skip to yearly menu bar Skip to main content


Poster
in
Workshop: New Frontiers in Associative Memories

Dense Associative Memory with Epanechnikov energy

Benjamin Hoover · Krishna Balasubramanian · Dmitry Krotov · Parikshit Ram


Abstract:

We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Uniquely, it introduces abundant additional emergent local minima while preserving perfect pattern recovery--a characteristic previously unseen in DenseAM literature. Empirical results show LSR generates significantly more local minima and produces samples with higher log-likelihood than LSE-based models, making it promising for both memory storage and generative tasks.

Chat is not available.