I have one zfs pool on my proxmox server with multiple datasets. Two datasets are passed through to two different identical virtual machines. I have an smb network share on the first vm mounted on the second vm. I'm transfering from the dataset on the first vm, to the one on the second using rsync.
rsync -rvtW --stats /source /dest
It will transfer at 150-300MB/s for about twenty seconds, before halting entirely for a minute and a half to two minutes, then returning to the previous speed.
I've done similar transfers in the past without any issues. The datasets have thin provisioning, and the receving dataset have primary and secondary cache set to metadata only. Am I doing something retarded here?