>want to download AI models on huggingface
>there's multiple checkpoints that i want to download, all on single repo totalling 40GB
>can't be fucked to download one by one
>i know a bit of git, so i just git clone the repo
>feelin hackerman when i type in the command prompt
>git clone finish
>holdup.jpg
>why is my hard drive is now 80GB less space?
>thats twice the repo size
>check the cloned repo
>it's 80GB
>wtf
>dig through the repo to find whats taking the extra space
>turns out there's a hidden .git\lfs folder
>turns out the folder contains the EXACT copy of the repo files, but with randomized names
>why, and what the fuck are these leftovers?
>turns out that this is a feature, not a bug of git ""LFS system"
>turns out that all of my other previous git clones were also doing this. taking up 200GB+ of wasted space
>mfw

anyone else knows this feel?

>>106150114 (OP)
git clone --depth 1

You can't be serious. So those 84949494 mysterious GB taking up my drive are.....shit that git clone created?
Fuck

>>106150114 (OP)
I only use shit that auto downloads from Pinokio for dummies, but I have 100tb, so no I don't.
Hugging face is also Purging models atm, so take note

>>106150114 (OP)
I learned about --depth 1 because I don't have internet at home and my mobile network is capped to 10 gb month.

>>106150206
isn't pinokio like 60gb?

>>106150196
doesnt work. it will still create .git\lfs folder for files tagged as LFS blobs

>>106150199
im serious. i've been using "git clone <repo url>" on huggingface for about a year now and i just realized about this LFS bullshit.

im just wondering if anyone else also just stumbled upon this predicament.

for any anons on Windows who downloaded repos on huggingface via git clone, try running Everything (the search app), turn on regex mode and search for:

\.git\\lfs$

>>106150264
yes, it does work. if you really don't need the .git folder, just delete the .git folder.
rm -r .git
and if you really really can't afford to spend the bandwidth downloading extra files in the first place, you shouldn't be using git clone in the first place.
git archive --remote=ssh://user@host/path/to/repo.git master
basically, you don't know what you're doing.

>>106150114 (OP)
Don't be a retard. git lfs is meant to capture all the changes made to the repo, it's going to be much larger than just using huggingface-cli download or wget to grab what you want.
Lots of split files to grab? You can use a bash one-liner with a for r in {1..99}; do... and grab them all. It will be much faster than using git clone.

>>106150357
>yes, it does work
not it doesnt. im talking about git lfs, not branches/commits. you can try yourself.
try on this 1GB repo with only a few files:

git clone --depth 1 https://huggingface.co/lllyasviel/control_v11f1e_sd15_tile


it will end up as a 2GB repo even with --depth 1

>and if you really really can't afford to spend the bandwidth
>basically, you don't know what you're doing.
i think you're the one not understanding the issue. and also, ironically, i've check my bandwith, it consumed just 1GB. im guess git clone will download files marked as lfs blobs into .git\lfs first, and then it goes into 'unpacking/filtering' process (as said on the progress prompt) and create the files with proper names as a copy

>>106150589
lmk when you finish reading my whole post