In attempting a one-off migratione of data from my existing homebrew NAS to TrueNAS, I popped open an SSH console and started a pull. Command was..
rsync -arvh --info=progress2 -e ssh --no-perms email@example.com:/public/. /mnt/tank/Storage/
After it ran a few minutes, I noticed that the speed was right around 18MB/s despite having a 10Gbe connection.
On a whim, I opened up multiple SSH windows, and set rsync to running in multiple subdirectories. They each stopped around 18MB/s.
I tried it with --bwlimit=500000, and it changed nothing.
Originally it was using compression as well, but since the spinning rust is the bottleneck, I took that off, no change.
I tried from a VM on an ESXi box, which was only a 1Gbe connection to the source server, and it immediately started pulling around 100MB/s, while TrueNAS rsync was pulling the 18MB/s many times over in other windows.
TrueNAS and the VM are both running rsync 3.1.3, the source server is running 3.1.2