Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Cloud Sync to Backblaze B2 fails

Description

I have a Cloud Sync setup to BackBlaze B2. It's about 4.5GB in size. Snapshot was not enabled.

The task seems to run forever.

Checking /tmp/middleward/jobs/ the last file sync'd is over over 12 hours ago.

When I clicked on the RUNNING label under Tasks > Cloud Sync it showed the attached image for over 10 minutes so the task appeared to be stuck.

I stopped the task and got this error:

Error: Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/middlewared/job.py", line 349, in run
await self.future
File "/usr/local/lib/python3.7/site-packages/middlewared/job.py", line 386, in __run_body
rv = await self.method(*([self] + args))
File "/usr/local/lib/python3.7/site-packages/middlewared/schema.py", line 961, in nf
return await f(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/cloud_sync.py", line 1007, in sync
await rclone(self.middleware, job, cloud_sync)
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/cloud_sync.py", line 218, in rclone
await asyncio.wait_for(check_cloud_sync, None)
File "/usr/local/lib/python3.7/asyncio/tasks.py", line 414, in wait_for
return await fut
File "/usr/local/lib/python3.7/site-packages/middlewared/plugins/cloud_sync.py", line 377, in rclone_check_progress
read = (await proc.stdout.readline()).decode()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 112: invalid start byte

I looked through /var/log/middlewared.log and could only find the entry where rclone starts and no errors after that.

Problem/Justification

None

Impact

None

is duplicated by

SmartDraw Connector

Katalon Manual Tests (BETA)

Activity

Show:

Vladimir VinogradenkoJuly 3, 2020 at 6:58 PM

the thread that was reading and reporting rclone progress was crashing (rclone also was getting stuck because it's stdout buffer was overflown), the patch above should fix it.

Howard HaJuly 3, 2020 at 6:32 PM

I thought I replied to this.

But the problem happens all the time.  I did some testing using rclone on FreeNAS command line and it doesn't seem to freeze even doing many files, but using the Cloud Sync UI some of the tasks will definitely just freeze and do nothing.  I have to cancel the task and start it again manually over and over again.

Bug ClerkJuly 2, 2020 at 2:26 PM

Bug ClerkJune 30, 2020 at 12:13 PM

Vladimir VinogradenkoJune 29, 2020 at 6:48 PM

did it happen again? Unfortunately we can't know what happened for the first time. Also I don't see any related fixed in rclone changelog, and we won't be updating rclone in stable branch because that has been breaking things too often in the past.

Complete

Details

Assignee

Reporter

Impact

High

Components

Fix versions

Affects versions

Priority

More fields

Katalon Platform

Created June 21, 2020 at 8:56 PM
Updated July 1, 2022 at 4:52 PM
Resolved July 2, 2020 at 2:26 PM