>>105614673
GQA and its devil offspring are the true killers of soulful outputs. This was common knowledge back during the llama2 era and the first command-r was good because it used natural attention as well