For anyone interested.
This fetches the model. It doesn't do a checkout of the weights, so it doesn't use double the storage. In addition, can resume downloads, verifies the files for you, it's easy to update files if anything changes in the main repo, you can see the history of changes, blablabla...
git clone ${repo}
git -C ${repo} lfs install --local
git -C ${repo} lfs fetch


If there are files you don't want to download, exclude them with
git -C ${repo} config --local lfs.fetchexclude "yourglobhere"


Save this somewhere. It links the regular and lfs files to their respective file in the actual repo. It's a smaller version of the script I typically use. Works fine with ksh. Bash should work just fine. Export dir needs to be in the same FS as the repo.
#export.sh
repo="$1"
output="$2"
mkdir ${output}
repo=$(realpath $repo)
output=$(realpath $output)

git -C ${repo}/ ls-files | while IFS= read ;do
f=$REPLY
mkdir -p "${output}/$(dirname $f)"
ln -s "${repo}/${f}" "${output}/${f}"
done

git -C ${repo}/ lfs ls-files -l | while IFS= read ;do
h=$(echo $REPLY | cut -f 1 -d " " )
f=$(echo $REPLY | cut -f 3 -d " " )
a=$(echo $h | cut -b 1,2 )
b=$(echo $h | cut -b 3,4 )
echo "$a/$b/$h -> $f"

mkdir -p "${output}/$(dirname $f)"
[ -h "${output}/${f}" ] && rm "${output}/${f}"
ln -s "${repo}/.git/lfs/objects/${a}/${b}/${h}" "${output}/${f}"
done


And run like
sh export.sh ${repo} ${repo}_export


Then convert normally from ${repo}_export.