Uploaded image for project: 'TrueNAS'
  1. TrueNAS
  2. NAS-109105

TrueNAS crash after VolumeStatus error

    XMLWordPrintable

    Details

    • Impact:
      High

      Description

      Hi Support

       

      This morning i woke to an email alert that my offsite backup TrueNAS server had had an unexpected restart. I've checked the report graphs for the UPS state but it doesn't appear to have been a power issue, I then noticed the following alarm...

      CRITICAL

      Failed to check for alert VolumeStatus: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/alert.py", line 706, in __run_source alerts = (await alert_source.check()) or [] File "/usr/local/lib/python3.8/site-packages/middlewared/alert/source/volume_status.py", line 31, in check for vdev in await self.middleware.call("pool.flatten_topology", pool["topology"]): File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1238, in call return await self._call( File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1206, in _call return await self.run_in_executor(prepared_call.executor, methodobj, *prepared_call.args) File "/usr/local/lib/python3.8/site-packages/middlewared/main.py", line 1110, in run_in_executor return await loop.run_in_executor(pool, functools.partial(method, *args, **kwargs)) File "/usr/local/lib/python3.8/site-packages/middlewared/utils/io_thread_pool_executor.py", line 25, in run result = self.fn(*self.args, **self.kwargs) File "/usr/local/lib/python3.8/site-packages/middlewared/plugins/pool.py", line 438, in flatten_topology d = deque(sum(topology.values(), [])) AttributeError: 'NoneType' object has no attribute 'values'

      2021-01-24 07:23:41 (Europe/London)

      The server reports that its uptime is  2:09 as of 09:32, which puts the restart at roughly the same time. I'm running the lastest 12.0-U1.1

      Server has 8x 8Tb WD Reds (Pre SMR-gate)

      2x SanDisk 256Gb SSDs (for VMs) Mirroed

      2x Integral SSDs mirrored as boot drives.

      Xeon E5-2630v4

      256Gb ECC memory.

      2x LSI 9217-8I in IT Mode

      The server sites idle throughout the day, and recieved replicated snapshots over a 200Mbps link from my primary server overnight, however these jobs are on hold as I'm rebuilding my primary site, so the server shouldn't have been doing anything.

       

      Please let me know if theres any logs/details I can pull to assist with troubleshooting or if theres any information you need that I'm missed.

       

      Thank you very much for your assistance in advance.

       

      Kind regards

      Rich

       

        Attachments

          Attachments

            JEditor

              Issue Links

                Activity

                  People

                  Assignee:
                  releng Release Council
                  Reporter:
                  Sprint Richard Green
                  Watchers:
                  Bonnie Follweiler, Richard Green, William Grzybowski
                  Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                    Dates

                    Created:
                    Updated:
                    Resolved: