Uploaded image for project: 'FreeNAS / TrueNAS'
  1. FreeNAS / TrueNAS
  2. NAS-107599

Pull replication is failing (12.0-BETA2.1 -> RC-1 regression)

    XMLWordPrintable

    Details

      JEditor

        Description

        Since upgrading from 12.0-BETA2.1 to RC-1 my pull replication job (which was previously working under BETA-2) has started failing with the below stack trace:

        [2020/09/16 20:28:12] INFO     [MainThread] [zettarepl.zettarepl] Scheduled tasks: [<Replication Task 'task_2'>]
        [2020/09/16 20:28:12] INFO     [Thread-9] [zettarepl.paramiko.replication_task__task_2] Connected (version 2.0, client OpenSSH_7.9p1)
        [2020/09/16 20:28:13] INFO     [Thread-9] [zettarepl.paramiko.replication_task__task_2] Authentication (publickey) successful!
        [2020/09/16 20:28:15] INFO     [replication_task__task_2] [zettarepl.replication.run] For replication task 'task_2': doing pull from 'storage/srv' to 'storage/incoming/srv' of snapshot='autosnap_2020-09-16_00:00:00_daily' incremental_base='autosnap_2020-09-15_00:00:00_da
        ily' receive_resume_token=None
        [2020/09/16 20:28:18] ERROR    [replication_task__task_2] [zettarepl.replication.run] For task 'task_2' unhandled replication error AttributeError("'SshReplicationProcess' object has no attribute 'stop_progress_observer'")
        Traceback (most recent call last):
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/run.py", line 145, in run_replication_tasks
            run_replication_task_part(replication_task, source_dataset, src_context, dst_context, observer)
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/run.py", line 219, in run_replication_task_part
            run_replication_steps(step_templates, observer)
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/run.py", line 490, in run_replication_steps
            replicate_snapshots(step_template, incremental_base, snapshots, observer)
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/run.py", line 540, in replicate_snapshots
            run_replication_step(step_template.instantiate(incremental_base=incremental_base, snapshot=snapshot), observer)
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/run.py", line 605, in run_replication_step
            ReplicationProcessRunner(process, monitor).run()
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/process_runner.py", line 33, in run
            raise self.process_exception
          File "/usr/local/lib/python3.8/site-packages/zettarepl/replication/process_runner.py", line 37, in _wait_process
            self.replication_process.wait()
          File "/usr/local/lib/python3.8/site-packages/zettarepl/transport/ssh.py", line 141, in wait
            self._stop_progress_observer()
          File "/usr/local/lib/python3.8/site-packages/zettarepl/transport/progress_report_mixin.py", line 47, in _stop_progress_observer
            if self.stop_progress_observer:
        AttributeError: 'SshReplicationProcess' object has no attribute 'stop_progress_observer'
        [2020/09/16 20:28:30] INFO     [Thread-11] [zettarepl.paramiko.retention] Connected (version 2.0, client OpenSSH_7.9p1)
        [2020/09/16 20:28:30] INFO     [Thread-11] [zettarepl.paramiko.retention] Authentication (publickey) successful!
        [2020/09/16 20:28:33] INFO     [retention] [zettarepl.zettarepl] Retention destroying local snapshots: []
        

        Nothing has changed on either system except the 12.0-BETA2.1 to RC-1 upgrade. The source system is configured as it always was (Debian Buster, ZFS-on-Linux 0.8.4), and the replication task is configured the same way it was before the upgrade, but since the upgrade it won't run.

          Attachments

          1. debug-truenas-20200916223922.tgz
            876 kB
          2. task_1.PNG
            task_1.PNG
            61 kB
          3. task_2.PNG
            task_2.PNG
            22 kB

            Attachments

              Issue Links

                Activity

                  People

                  Assignee:
                  releng Release Council
                  Reporter:
                  gac Gavin Chappell
                  Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                    Dates

                    Created:
                    Updated:
                    Resolved: