Search Results
6/11/2025, 11:21:54 PM
>However, for the full R1-0528 model which is 715GB in size, you will need extra prep. The 1.78-bit (IQ1_S) quant will fit in a 1x 24GB GPU (with all layers offloaded). Expect around 5 tokens/s with this setup if you have bonus 128GB RAM as well.
https://docs.unsloth.ai/basics/deepseek-r1-0528-how-to-run-locally
ALL layers into mere 24 GB?????
https://docs.unsloth.ai/basics/deepseek-r1-0528-how-to-run-locally
ALL layers into mere 24 GB?????
Page 1