Search Results

Found 1 results for "5b93984eb3b8f4fa0db076f9fa64a8ef" across all boards searching md5.

Anonymous /g/105689385#105694431
6/25/2025, 12:08:24 AM
ahoy we have a liftoff. just with

./llama.cpp/build/bin/llama-cli \
--rpc "$RPC_SERVERS" \
--model models/unsloth/DeepSeek-R1-0528-GGUF/UD-Q2_K_XL/DeepSeek-R1-0528-UD-Q2_K_XL-00001-of-00006.gguf \
--cache-type-k q4_0 \
--threads 48 \
--n-gpu-layers 99 \
--prio 3 \
--temp 0.6 \
--top_p 0.95 \
--min_p 0.01 \
--ctx-size 16384 \
-ot ".ffn_(up)_exps.=CPU" \
-no-cnv

but i still have about 4gb free per gpu, i can probably only offlead thee last 20 or so layers.
ill report back