>>8624863
seconding >>8625207, i'm interested to know how your are going about your finetuning; i've got some questions too
1) do you have a custom training script or are you using an existing one?
2) what is the training config you have setup for your finetuning, and is there any particular factors that made you consider those hyperparameters?
3) in terms of data preparation, is the prep for finetuning different from training loras? do you do anything special with the dataset?
4) i too am using a 3090 >>8625025, how much vram usage are you running at when performing a finetune at your current batch size?