We're seeing that customer systems have ever growing sqlite databases (freenas-v1.db). While investigation is on-going to see if this is expected, we're running into a minor problem that has a profound impact.
failover.sync_to_peer method uses the filesystem.file_receive method which uses Str class in the middlewared schema. This particular schema class has an kwarg "max_length" that is set to 2MB. The reason why this is set is because sqlite3 has a 2MB size limit for text columns. However, the filesystem.file_receive method is not talking to the database but is instead base64.encoding() the contents of the sqlite3 database contents. While this worked for us in the past, this is a moving target for when the database grows above the "max_length" kwarg for that class. Instead of using filesystem.file_receive, move to using failover.sendfile which doesn't have this artificial limitation.