Hacker News with Generative AI: AMD GPUs

Fixing sleep-wake hangs on Linux with AMD GPUs (gitlab.io)
I dual-boot my desktop between Windows and Linux. Over the past few years, Linux would often crash when I tried to sleep my computer with high RAM usage. Upon waking it would show a black screen with moving cursor, or enter a "vegetative" state with no image on-screen, only responding to magic SysRq or a hard reset. I traced this behavior to an amdgpu driver power/memory management bug, which took over a year to brainstorm and implement solutions for.

Linux, AMD GPUs, Debugging, Hardware, Software

13 points by dralley 18 days ago | 0 comments

Making AMD GPUs competitive for LLM inference (2023) (mlc.ai)
MLC-LLM makes it possible to compile LLMs and deploy them on AMD GPUs using ROCm with competitive performance.

Generative AI, Hardware, AMD GPUs, Performance

313 points by plasticchris 25 days ago | 213 comments

AMD GPU Inference (github.com/slashml)
This project provides a Docker-based inference engine for running Large Language Models (LLMs) on AMD GPUs. It's designed to work with models from Hugging Face, with a focus on the LLaMA model family.

Generative AI, AMD GPUs, Docker, Inference

270 points by fazkan 108 days ago | 99 comments