Pools going offline and then coming back almost immediately

Description

Truenas Scale:

I keep getting alerts:

New alerts: [LIST] [*]Pool GoodHDD state is OFFLINE: None [*]Pool SingleSSD state is OFFLINE: None /LIST Current alerts: [LIST] [*]Pool GoodHDD state is OFFLINE: None [*]Pool SingleSSD state is OFFLINE: None /LIST

Here are my middlware logs at the same time it happened:

[2022/06/18 04:32:34] (WARNING) middlewared._loop_monitor_thread():1625 - Task seems blocked:
File "/usr/lib/python3/dist-packages/middlewared/schema.py", line 1129, in nf
res = await f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 431, in is_upgraded
return await self.is_upgraded_by_name((await self.get_instance(oid))['name'])
File "/usr/lib/python3/dist-packages/middlewared/plugins/pool.py", line 448, in is_upgraded_by_name
proc = await Popen(
File "/usr/lib/python3.9/asyncio/subprocess.py", line 216, in create_subprocess_shell
transport, protocol = await loop.subprocess_shell(
... + 10 lines below ...
[2022/06/18 04:33:59] (DEBUG) google_auth_httplib2._call_():115 - Making request: POST https://oauth2.googleapis.com/token

Problem/Justification

None

Impact

None

Activity

Show:

Bonnie Follweiler 
June 23, 2022 at 1:02 PM

Thank you .

I have moved this ticket into our queue to review.

An engineering representative will update with any further questions or details in the near future.

Adam Carpentieri 
June 21, 2022 at 11:22 PM

Please find attached debug.

Bonnie Follweiler 
June 21, 2022 at 9:24 PM

Thank you for the report, .

Can you please attach a debug file to the "Private Attachments" section of this ticket?

To generate a debug file on TrueNAS SCALE, log in to the TrueNAS web interface, go to System Settings > Advanced, then click Save Debug and wait for the file to download to your local system.

Duplicate

Details

Assignee

Reporter

Labels

Components

Fix versions

Affects versions

Priority

More fields

Katalon Platform

Created June 19, 2022 at 4:40 PM
Updated July 6, 2022 at 8:59 PM
Resolved June 27, 2022 at 5:25 PM