>>106492238
On the one hand, rebooting and not running 5 other things helped.
On the other hand, noticed that llama.cpp now doesn't refuse to offload more layers to GPU than I have VRAM available. I guess it transparently offloads those layers again from GPU VRAM back to usual RAM, which kills the performance.
Either pulling llama.cpp repo or updating system drivers caused this, go figure.
Life of an AMD vramlet is full of pain and misery.