>>938807434
it's unorthodox that's for sure
just make sure you got enough VRAM, and use the correct CLIP version for the model you aim
SDXL uses both CLIP-G and CLIP-L, but I mostly use CLIP-G to generate the negative prompt