Search Results
7/17/2025, 10:05:59 PM
>>8663246
Thanks for the info.
>Dataset matters way more than settings,
Ah, damn. Prepping a dataset is the most pain in the ass part of training a LORA. It seems like you need to do a lot of manual tagging and image editing if you want a really good dataset.
>CAME with rex warm-annealing restarts is what anons mentioned with the 300 steps thing. Together with gradient accumulation which simulates a higher batch size, it's supposed to train very fast.
That is good to know! I guess I was under the impression that something like Prodigy would have a lot more value than it apparently does. I've also gotten results with AdamW that were workable at least
Maybe I'll have to try using CAME instead to see how much of a difference it makes. Never would have found out about it if not for /hgg/ , so thanks to the anons who still actually hang out here and make it worthwhile!
Thanks for the info.
>Dataset matters way more than settings,
Ah, damn. Prepping a dataset is the most pain in the ass part of training a LORA. It seems like you need to do a lot of manual tagging and image editing if you want a really good dataset.
>CAME with rex warm-annealing restarts is what anons mentioned with the 300 steps thing. Together with gradient accumulation which simulates a higher batch size, it's supposed to train very fast.
That is good to know! I guess I was under the impression that something like Prodigy would have a lot more value than it apparently does. I've also gotten results with AdamW that were workable at least
Maybe I'll have to try using CAME instead to see how much of a difference it makes. Never would have found out about it if not for /hgg/ , so thanks to the anons who still actually hang out here and make it worthwhile!
Page 1