someone hooked up a small llm as a text encoder for sdxl
https://huggingface.co/Minthy/RouWei-Gemma
https://github.com/NeuroSenko/ComfyUI_LLM_SDXL_Adapter
comfy only but it's quite fun to play with