More Gemma news.
https://huggingface.co/collections/google/t5gemma-686ba262fe290b881d21ec86
They trained Gemma from decoder only to be encoder decoder like T5. I am guessing this is where people are going to flock to for text encoders for new image diffusion models but for the most part, it seems like LLM and a projection layer might be better and more flexible.