On a Mac, I have a folder named "AUX.2016.03.09" with 61 files within. If I copy it (using sudo cp -a) to an SMB share on my FreeNAS 11.2-U7:
1) cp on the Mac reports various "No such file or directory" errors.
2) The resulting folder name gets mangled to "AG7JL4~7.09" (an 8.3 name?) when viewed from Finder over SMB. When ssh'ed to the FreeNAS, the folder name on disk is correct.
3) Only one of its 61 files is copied (either viewed over SMB or on the disk itself)
4) Trying to open the folder from Finder results in the folder disappearing.
Happily I can repro this. I increased Samba logging to "debug" and attach the logs here, as well as the command used on the Mac and its output.