Living Lab Lecture No. 30: Compressing Large Language Models

Datum

06.03.2025

Zeit

11:00 - 12:00

Sprecher

Aaron Klein

Zugehörigkeit

ScaDS.AI Dresden/Leipzig

Serie

ScaDS.AI Lecture Series

Sprache

en

Hauptthema

Informatik

Host

ScaDS.AI Dresden/Leipzig

Beschreibung

Large Language Models (LLMs) mark a new era in Artificial Intelligence. However, their large size poses significant challenges for inference in real-world applications due to substantial GPU memory requirements and high inference latency. In this talk, we discuss techniques to compress pre-trained LLMs, reducing their resource consumption during inference while maintaining their performance. More specifically, we approach the problem from a multi-objective Neural Architecture Search (NAS) perspective to jointly optimize performance and efficiency. By considering the LLM as a super-network consisting of a large but finite number of sub-networks, we can identify a set of Pareto-optimal sub-networks that balance parameter count and validation performance. We empirically demonstrate that using NAS techniques for fine-tuning enhances the prunability of pre-trained LLMs and explore how this impacts real-world applications.

Links

• More information

Letztmalig verändert: 17.02.2025, 16:08:07