Dense Associative Memory: physical systems for novel AI architectures

Date

Mar 23, 2026

Time

3:30 PM - 4:30 PM

Speaker

Prof Dr Dmitry Krotov

Series

MPI-PKS Kolloquium

Language

en

Main Topic

Physik

Other Topics

Physik

Description

Dense Associative Memories are recurrent neural networks with fixed point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in the 1980s, Dense Associative Memories have a very large memory storage capacity, which makes them appealing tools for many problems in AI and neuroscience. In this talk, I will provide an intuitive understanding and a mathematical framework for this class of models, and will give examples of problems in AI that can be tackled using these new ideas. Specifically, I will explore the relationship between Dense Associative Memories and two prominent generative AI models: transformers and diffusion models. I will present a neural network, called the Energy Transformer, which unifies energy-based modeling, associative memories, and transformers in a single architecture. Furthermore, I will discuss an emerging perspective that views diffusion models as Dense Associative Memories operating above the critical memory storage capacity. This insight opens up interesting avenues for leveraging associative memory theory to analyze the memorization-generalization transition in diffusion models, which is closely related to spin glass transition in Dense Associative Memories.

Last modified: Mar 2, 2026, 7:39:40 AM