Search Results
7/24/2025, 2:23:59 PM
>>106005673
https://www.arxiv.org/pdf/2506.03296
I ran across this and I've been wondering if anyone can spoon feed me which doc to go reading for effective use of using a GPU as an assist or cpu and ram as an assist. I've heard there are various methods now like ZeRO and such but I have no idea what is ideal or not.
I have 1 4090 and a 5945wx. I'd like to try running a local model but Its always felt like a waste to run something as small as can fit on 24 vram, but also I've never been sure how to configure it to be more than dead weight in my system if i use the cpu and dram. and now it appears between this white paper released in july 10 and ZeRO it might be better to use both at once in unfamiliar ways such as using the 4090 as the main processing unit and the CPU hands it slices of model which i thought was supposed to be impossible or pointless.
https://www.arxiv.org/pdf/2506.03296
I ran across this and I've been wondering if anyone can spoon feed me which doc to go reading for effective use of using a GPU as an assist or cpu and ram as an assist. I've heard there are various methods now like ZeRO and such but I have no idea what is ideal or not.
I have 1 4090 and a 5945wx. I'd like to try running a local model but Its always felt like a waste to run something as small as can fit on 24 vram, but also I've never been sure how to configure it to be more than dead weight in my system if i use the cpu and dram. and now it appears between this white paper released in july 10 and ZeRO it might be better to use both at once in unfamiliar ways such as using the 4090 as the main processing unit and the CPU hands it slices of model which i thought was supposed to be impossible or pointless.
Page 1