Search Results
6/14/2025, 8:25:20 PM
>>101191120
70B models actually run okay at 2.4bpw (roughly q2 here) which is 24 gigs of RAM/VRAM.
An anon got acceptable speeds speeds with Qwen3-30B at 8_0 (which is 32 gigs of RAM/VRAM) with a 3060 (12 gb VRAM) + offload. (https://desuarchive.org/g/thread/105125022/#105129861)
There's definitely a bit of wiggleroom to adjust your speeds.
70B models actually run okay at 2.4bpw (roughly q2 here) which is 24 gigs of RAM/VRAM.
An anon got acceptable speeds speeds with Qwen3-30B at 8_0 (which is 32 gigs of RAM/VRAM) with a 3060 (12 gb VRAM) + offload. (https://desuarchive.org/g/thread/105125022/#105129861)
There's definitely a bit of wiggleroom to adjust your speeds.
Page 1