4chan Search
2 results for "7bb8641d83d18b47a9cfd0bd771aea40"
>>106975949
After some trial and error, I can safely say don't bother trying to run this if you only have 48gb vram. You will OOM trying to load the model. I'm going to try the smaller 30B version tomorrow.
2x3090
https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8/tree/main
vllm serve . --port 8100 --max-model-len 2048 --tensor-parallel-size 2
After some trial and error, I can safely say don't bother trying to run this if you only have 48gb vram. You will OOM trying to load the model. I'm going to try the smaller 30B version tomorrow.
2x3090
https://huggingface.co/Qwen/Qwen3-VL-32B-Instruct-FP8/tree/main
vllm serve . --port 8100 --max-model-len 2048 --tensor-parallel-size 2