Dense Associative Memory: physical systems for novel AI architectures

Datum

23.03.2026

Zeit

15:30 - 16:30

Sprecher

Prof Dr Dmitry Krotov

Serie

MPI-PKS Kolloquium

Sprache

en

Hauptthema

Physik

Andere Themen

Physik

Beschreibung

Dense Associative Memories are recurrent neural networks with fixed point attractor states that are described by an energy function. In contrast to conventional Hopfield Networks, which were popular in the 1980s, Dense Associative Memories have a very large memory storage capacity, which makes them appealing tools for many problems in AI and neuroscience. In this talk, I will provide an intuitive understanding and a mathematical framework for this class of models, and will give examples of problems in AI that can be tackled using these new ideas. Specifically, I will explore the relationship between Dense Associative Memories and two prominent generative AI models: transformers and diffusion models. I will present a neural network, called the Energy Transformer, which unifies energy-based modeling, associative memories, and transformers in a single architecture. Furthermore, I will discuss an emerging perspective that views diffusion models as Dense Associative Memories operating above the critical memory storage capacity. This insight opens up interesting avenues for leveraging associative memory theory to analyze the memorization-generalization transition in diffusion models, which is closely related to spin glass transition in Dense Associative Memories.

Letztmalig verändert: 02.03.2026, 07:39:40