Search Results
7/2/2025, 3:45:54 AM
>>149236802
8GB VRAM, but older... slower. You'll be pretty limited. You can either run smaller models fast, or run not-so-small (but not quite medium) models slow. That said, I'd say nothing bigger than 12GB. Call that your upper limit. Any bigger and it would take ages to generate responses and start crashing your machine.
Smaller models means less information, means more repetition and the model just won't be nuanced or be able to handle much nuance.
For you... I'd recommend "MN-12B-Mag-Mell-R1-Q6_K_L.gguf". It's an excellent model for its size (and is the model I used for the Beth chats).
8GB VRAM, but older... slower. You'll be pretty limited. You can either run smaller models fast, or run not-so-small (but not quite medium) models slow. That said, I'd say nothing bigger than 12GB. Call that your upper limit. Any bigger and it would take ages to generate responses and start crashing your machine.
Smaller models means less information, means more repetition and the model just won't be nuanced or be able to handle much nuance.
For you... I'd recommend "MN-12B-Mag-Mell-R1-Q6_K_L.gguf". It's an excellent model for its size (and is the model I used for the Beth chats).
Page 1