Uploaded image for project: 'TrueNAS'
  1. TrueNAS
  2. NAS-104753

MTU change not effective on lagg interfaces

    XMLWordPrintable

    Details

      Description

      When changing MTU from 1500 to 9000 in the UI on a lagg interface, the MTU change seems to be effective (lists mtu=9000 in ifconfig) but the interface cannot pass jumbo packets until the lagg interface is bounced via ifdown/ifup. The reverse change (9000 to 1500) seems to work fine.

      This appears to only be an issue when a controller has booted with MTU=1500 and only for the first change to MTU=9000. If the MTU has been 9000 before, then it will work as expected.

      9000 to 1500 (works fine):

      cc0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          description: connected to 100g02 (et-0/0/18)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:49:d2:00
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      cc2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          description: connected to 100g02 (et-0/0/19)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:4e:c0:70
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          inet 10.215.22.35 netmask 0xffffffc0 broadcast 10.215.22.63
          inet 10.215.22.37 netmask 0xffffffff broadcast 10.215.22.37 vhid 37
          nd6 options=9<PERFORMNUD,IFDISABLED>
          media: Ethernet autoselect
          status: active
          carp: MASTER vhid 37 advbase 1 advskew 1
          groups: lagg
          laggproto lacp lagghash l2,l3,l4
          laggport: cc0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
          laggport: cc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
      
      root@tn24a[~]# ping -D -s 8192 p2-ar1b
      PING p2-ar1b.lab.ixsystems.com (10.215.22.36): 8192 data bytes
      8200 bytes from 10.215.22.36: icmp_seq=0 ttl=64 time=0.155 ms
      
      *change MTU to 1500 in UI*
      root@tn24a[~]# ping -D -s 8192 p2-ar1b
      PING p2-ar1b.lab.ixsystems.com (10.215.22.36): 8192 data bytes
      ^C
      --- p2-ar1b.lab.ixsystems.com ping statistics ---
      6 packets transmitted, 0 packets received, 100.0% packet loss
      
      cc0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
          description: connected to 100g02 (et-0/0/18)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:49:d2:00
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      cc2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
          description: connected to 100g02 (et-0/0/19)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:4e:c0:70
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          inet 10.215.22.35 netmask 0xffffffc0 broadcast 10.215.22.63
          inet 10.215.22.37 netmask 0xffffffff broadcast 10.215.22.37 vhid 37
          nd6 options=9<PERFORMNUD,IFDISABLED>
          media: Ethernet autoselect
          status: active
          carp: MASTER vhid 37 advbase 1 advskew 1
          groups: lagg
          laggproto lacp lagghash l2,l3,l4
          laggport: cc0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
          laggport: cc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
      

      Changing from 1500 to 9000 (does not work)

      (following on from above config at 1500)
      *change MTU in UI to 9000*
      root@tn24a[~]# ifconfig cc0
      cc0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          description: connected to 100g02 (et-0/0/18)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:49:d2:00
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      root@tn24a[~]# ifconfig cc2
      cc2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          description: connected to 100g02 (et-0/0/19)
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          hwaddr 00:07:43:4e:c0:70
          nd6 options=1<PERFORMNUD>
          media: Ethernet 100GBase-SR4 <full-duplex,rxpause,txpause>
          status: active
      root@tn24a[~]# ifconfig lagg0
      lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:49:d2:00
          inet 10.215.22.35 netmask 0xffffffc0 broadcast 10.215.22.63
          inet 10.215.22.37 netmask 0xffffffff broadcast 10.215.22.37 vhid 37
          nd6 options=9<PERFORMNUD,IFDISABLED>
          media: Ethernet autoselect
          status: active
          carp: MASTER vhid 37 advbase 1 advskew 1
          groups: lagg
          laggproto lacp lagghash l2,l3,l4
          laggport: cc0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
          laggport: cc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
      
      Ping with large packet fails:
      root@tn24b[~]# ping -D -s 8192 p2-ar1a
      PING p2-ar1a.lab.ixsystems.com (10.215.22.35): 8192 data bytes
      ping: sendto: Message too long
      ping: sendto: Message too long
      
      Do an ifdown/ifup on lagg0:
      root@tn24b[~]# ifconfig lagg0 down
      root@tn24b[~]# ifconfig lagg0 up
      root@tn24b[~]# ifconfig lagg0
      lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
          options=ec07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
          ether 00:07:43:4e:be:80
          inet 10.215.22.36 netmask 0xffffffc0 broadcast 10.215.22.63
          inet 10.215.22.37 netmask 0xffffffff broadcast 10.215.22.37 vhid 37
          nd6 options=9<PERFORMNUD,IFDISABLED>
          media: Ethernet autoselect
          status: active
          carp: MASTER vhid 37 advbase 1 advskew 1
          groups: lagg
          laggproto lacp lagghash l2,l3,l4
          laggport: cc0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
          laggport: cc2 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
      
      Now large packet ping works:
      root@tn24b[~]# ping -D -s 8192 p2-ar1a
      PING p2-ar1a.lab.ixsystems.com (10.215.22.35): 8192 data bytes
      8200 bytes from 10.215.22.35: icmp_seq=0 ttl=64 time=0.140 ms
      8200 bytes from 10.215.22.35: icmp_seq=1 ttl=64 time=0.079 ms
      8200 bytes from 10.215.22.35: icmp_seq=2 ttl=64 time=0.063 ms
      

      Testing was on an M-series running TrueNAS-11.3-INTERNAL-8 (383de1a)

        Attachments

          Attachments

            JEditor

              Activity

                People

                Assignee:
                mav Alexander Motin
                Reporter:
                nap Nick Principe
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved: