Uploaded image for project: 'FreeNAS / TrueNAS'
  1. FreeNAS / TrueNAS
  2. NAS-106928

zettarepl middlewared file descriptor leak

    XMLWordPrintable

    Details

    • Impact:
      Medium

      JEditor

        Description

        Checking middlewared/zettarepl process shows file descriptor leak over time, which on our production system ended up eating all pipekva space. We got receiving the following kernel messages:

        kern.ipc.maxpipekva exceeded; see tuning(7)
         

        Restarting middlewared solves the problem temporary, but over time resources will again be exhausted.

        The leak can be observer:

         

        nas-a# ps axw|grep middlewared.*zettar
        86105  -  S         0:23.00 python3.7: middlewared (zettarepl) (python3.7)
        91844  1  S+        0:00.00 grep middlewared.*zettar
        nas-a# procstat -f 86105 | wc -l
             431
        nas-a# procstat -f 86105 | wc -l
             433
        nas-a# procstat -f 86105 | wc -l
             433
        nas-a# procstat -f 86105 | wc -l
             433
        nas-a# procstat -f 86105 | wc -l
             434
        nas-a# procstat -f 86105 | wc -l
             434
        nas-a# procstat -f 86105 | wc -l
             434
        nas-a# procstat -f 86105 | wc -l
             435
        nas-a# procstat -f 86105 | wc -l
             436
        nas-a# procstat -f 86105 | wc -l
             436
        nas-a# procstat -f 86105 | wc -l
             436
        nas-a# procstat -f 86105 | wc -l
             436
        nas-a# procstat -f 86105 | wc -l
             437
        nas-a# procstat -f 86105 | wc -l
             437
         

        This has been captured during a replication task running, while sending multiple snapshots.

          Attachments

            Attachments

              Issue Links

                Activity

                  People

                  Assignee:
                  themylogin Vladimir Vinogradenko
                  Reporter:
                  krichy Richard Kojedzinszky
                  Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                    Dates

                    Created:
                    Updated:
                    Resolved:

                      Summary Panel