Thanks for using the TrueNAS Community Edition issue tracker! TrueNAS Enterprise users receive direct support for their reports from our support portal.

Kerberized NFS not working after upgrade to 12.0U5+

Description

After upgrading TrueNAS from 12.0U4.1 to 12.0U5 NFSv4 mounts are not accessible anymore with sec=krb5/p/i.

Running `gssd -v -d` manually shows no output at all when trying to mount a nfs share from a client. Clients do not receive a nfs service ticket though it exists in the TrueNAS keytab.

Doing the same with 12.0U4.1 shows output and mounts successfully:

gssd -v -d
gssd_accept_sec_context: cred resource not found
gssd_import_name: done major=0x0 minor=0
gssd_release_cred: done major=0x0 minor=0
gssd_acquire_cred: done major=0x0 minor=0
gssd_release_name: done major=0x0 minor=0
gssd_accept_sec_context: done major=0x0 minor=0
gssd_export_sec_context: done major=0x0 minor=0
gssd_export_name: done major=0x0 minor=0
gssd_pname_to_uid: failed major=0xd0000 minor=-1765328227

More information:

https://www.truenas.com/community/threads/nfs4-broken-after-upgrade-to-truenas-12.89983/post-657160

https://www.truenas.com/community/threads/cant-access-nfsv4-shares-after-upgrade-to-truenas-12-0-u5.94543/

Problem/Justification

None

Impact

None

Activity

Devin Kusek 
April 13, 2022 at 3:20 PM

 

New case has come up in regards to this. 
User tried update to  U8 and needed to roll back

Debug from that case once the customer updated (debug is from U8 but they have rolled back to 11.3-U5)

"""
We had nfs w/ krb5 working under 11.3. After upgrading to 12.0 it no longer works. I reissued the keytab and can successfully "kinit -k" but nfsd fails to register with gssd. I tried running gssd in verbose and debug modes, but nothing was logged.

"""

""Running `gssd -v -d` manually shows no output at all""

Do we have anything additionally we can do? 

Caleb 
February 8, 2022 at 7:10 PM

Unfortunately, we do not have time or resources to track this down. Kerberized NFSv4 is barely used in the wild so there aren't too many eyes looking at this code base. I would suggest opening a ticket with upstream freeBSD to see if they can help resolve this problem. The other option is to try and test SCALE since we have confirmed that kerberized NFS is working there.

Andreas 
November 18, 2021 at 7:46 AM

After some further testing, I found that not only kerberized NFSv4 is no longer working, everything other than NFSv3 fails.

Maybe https://ixsystems.atlassian.net/browse/NAS-108869#icft=NAS-108869 is related too.

Andreas 
November 17, 2021 at 4:47 PM

I'm stuck with the same issue and before reverting back to 12.0-U4, I wanted to share some information about `gssd` which seem to be curlpit here:

I followed the steps from here, but added `truss` to start `gssd`

Hopefully the information helps solving that issue.

Pascal Pascher 
October 26, 2021 at 3:19 PM

I added two more private attachments:

12u6.pcap and 12u41.pcap - Packet Capture on the freenas host (tcpdump -s 0 -w <filename>.pcap host <client>) while attempting a "sudo mount -vvvvv -o vers=4.1 freenas01:/mnt/tank/media /mnt/freenas/media/" from a CentOS 8 Stream Client once to TrueNAS 12u6 and once to 12u4.1 respectively.

There are some differences with the initial "NFS V4 NULL Call" packets between the different TrueNAS versions for the RPC part it seems.

12u6 seems to attempt rpcsec_gss_init procedure multiple times before giving up.

 

Third Party to Resolve

Details

Assignee

Reporter

Labels

Time remaining

0m

Fix versions

Affects versions

Priority

Katalon Platform

Created September 1, 2021 at 9:05 PM
Updated July 6, 2022 at 8:56 PM
Resolved February 8, 2022 at 7:10 PM