Replication progress % Incorrect

20.12 PR: https://github.com/freenas/freenas/pull/5876

12.0 PR: https://github.com/freenas/freenas/pull/5833

20.10 PR: https://github.com/freenas/zettarepl/pull/118

Former user Is that the current algorithm used or a proposed algorithm? If it's a proposed algorithm, then if I have three snapshots to send for a dataset, and I'm 50% of the way through the second snapshot, my rep progress = (1+50/100)/3 = 50%? First impressions might be that this would seem like a reasonable approach. If you're suggesting this is the current algorithm, then I would say the evidence is that it's not a reasonable approach. Why?

The issue with the algorithm is that it does not take into account the size of the dataset. Consider, for example, a parent root A with a pair of nested datasets (B & C). Let's say the bulk of the data is in child dataset C and there's negligible data in A and B. Assuming there are no previous snapshots, using the algorithm above, and we're 50% through C, the calculated rep progress = (2+.5)/3 = 83%. However, in reality, if there's almost zero data in A & B, the rep progress is actually closer to 50%.

Similarly, if we consider just one dataset, with no previous snapshots, let's now assume three snapshots A, B & C are created for it. A replication is triggered after all three snapshots are created. For a very large dataset, snapshot A will take the longest to replicate. Assume there's no change to the dataset so B & C will be empty and we're 50% through replicating A. Using the algorithm above, rep progress = .5/3 = 17%. In actual fact, it's closer to 50%.

 Also "snapshot progress" is only available for SSH transport, not SSH+NETCAT, what transport are you using?

That's surprising, because what you're suggesting is that the ability to calculate rep progress is dependent on the transport used. I'm using SSH+NETCAT and based on my observations within the forum, this seems to be a popular choice.  

replication progress is calculated as (number of snapshots sent + current snapshot progress / 100) / (total number of snapshots), ZFS provides no other way to estimate the amount of data to be sent. Does this fit your observations? Also "snapshot progress" is only available for SSH transport, not SSH+NETCAT, what transport are you using?