@aoxo
I'm an AI researcher that loves making and breaking LLMs
We burned 48 GPU hours finding out how few bits a model needs
Alosh Denny, Senior Staff Engineer at Shield AI
Birds of Feather (BOF) session: Fewer bits, more questions: making quantization work across models, runtimes and hardware
Facilitated by Alosh Denny and Vivek Kalyanarangan