samba - Nov 2019 - Recent inability to view long filenames stored with scp via samba mount

Greetings Jeremy,

Jeremy Allison wrote:
> On Sun, Nov 03, 2019 at 06:37:57AM -0500, Michael Lueck via samba wrote:
>>
>>
>> mdlueck at
ldslnx01:/srv/shares/data/Nodes/jacob/home/mdlueck/Music/Alfred Brendel$ find
>> .
>> ./Beethoven Piano Sonatas Vol I
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 01 - Piano Sonata No. 29 in
B-flat major, op. 106 "Hammerklavier": I. Allegro.mp3
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 02 - Piano Sonata No. 29 in
B-flat major, op. 106 "Hammerklavier": II. Scherzo.mp3
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 03 - Piano Sonata No. 29 in
B-flat major, op. 106 "Hammerklavier": III. Adagio sostenuto.mp3
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 04 - Piano Sonata No. 29 in
B-flat major, op. 106 "Hammerklavier": IV: Largo; Allegro risoluto.mp3
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 05 - Piano Sonata No. 32 in C
minor, op. 111: I. Maestoso; Allegro con brio e appassionato.mp3
>> ./Beethoven Piano Sonatas Vol I/Disc 1 - 06 - Piano Sonata No. 32 in C
minor, op. 111: II. Arietta; Adagio molto, semplice e cantabile.mp3
>>
>>
>> Good!
>>
>>
>> View across the mount.cifs share to my workstation...
>>
>>
>> mdlueck at
jacob:/mnt/ldslnx01/data/Nodes/jacob/home/mdlueck/Music/Alfred Brendel$ find
>> .
>> ./Beethoven Piano Sonatas Vol I
>> ./Beethoven Piano Sonatas Vol I/DTCML9~Y.MP3
>> ./Beethoven Piano Sonatas Vol I/DQIP6H~S.MP3
>> ./Beethoven Piano Sonatas Vol I/DKU9ZS~P.MP3
>> ./Beethoven Piano Sonatas Vol I/D4K1AV~O.MP3
>> ./Beethoven Piano Sonatas Vol I/DIVFAH~2.MP3
>> ./Beethoven Piano Sonatas Vol I/DWMU18~G.MP3
>> ./Beethoven Piano Sonatas Vol I/DDDAFP~U.MP3
>> ./Beethoven Piano Sonatas Vol I/DIYJ3G~3.MP3
>> ./Beethoven Piano Sonatas Vol I/DQYYDL~5.MP3
>>
>>
>>
>> Yikes!! Why is Samba not showing the long version of file names
actually successfully on the server suddenly?
>>
>> Did something go wrong after yesterday's update?
> Nope. Your filenames have the ':' character in them,
> which is invalid for SMB2 filenames. So Samba
> must mangle them to make them available to SMB2
> clients.

Indeed, I was finding other exceptions than just the new Beethoven Piano Sonata
files. Some dating back to 2012! So definitely not new. I just have not needed
to swap disks on my workstation in so
long I have not needed to restore the MP3 collection.

Both the : and the ? characters are key to having Samba limit the filename to
the short version. I took care to remove both chars (one at a time) from a MP3,
rync to backup, and once both chars were
removed, NOW I can see the same long filename via the SAMBA mount as I see on
the native Linux file system.

Bravo Jeremy! Excellent sleuthing all!

So what is a complete list of not allowable chars for SMB compatibility? I
intent to purge my MP3 collection of all invalid chars.

I am thankful,

-- 
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/

On Fri, Nov 08, 2019 at 03:20:14PM -0500, Michael Lueck via samba
wrote:
> 
> Indeed, I was finding other exceptions than just the new Beethoven Piano
> Sonata files. Some dating back to 2012! So definitely not new. I just have
> not needed to swap disks on my workstation in so long I have not needed to
> restore the MP3 collection.
> 
> Both the : and the ? characters are key to having Samba limit the filename
> to the short version. I took care to remove both chars (one at a time) from
> a MP3, rync to backup, and once both chars were removed, NOW I can see the
> same long filename via the SAMBA mount as I see on the native Linux file
> system.
Yep, I had to do this to my media collection too
in order to move to SMB2 mounts for better
performance.
> Bravo Jeremy! Excellent sleuthing all!
> 
> So what is a complete list of not allowable chars for SMB compatibility? I
intent to purge my MP3 collection of all invalid chars.
In the code it's:

        for (i=1;i<128;i++) {
                if (i <= 0x1f) {
                        /* Control characters. */
                        char_flags[i] |= FLAG_ILLEGAL;
                }

		....

                if (strchr("*\\/?<>|\":", i)) {
                        char_flags[i] |= FLAG_ILLEGAL;
                }

		....
        }


but we must also mangle names ending in '.' and any
names that match DOS reserved names of:

static const char * const reserved_names[] { "AUX", "LOCK$",
"CON", "COM1", "COM2", "COM3",
"COM4",
  "LPT1", "LPT2", "LPT3", "NUL",
"PRN"

Not that your mp3 files would match those :-).

Greetings Jeremy,


Jeremy Allison wrote:
> In the code it's:
> 
>          for (i=1;i<128;i++) {
>                  if (i <= 0x1f) {
>                          /* Control characters. */
>                          char_flags[i] |= FLAG_ILLEGAL;
>                  }
> 
> 		....
> 
>                  if (strchr("*\\/?<>|\":", i)) {
>                          char_flags[i] |= FLAG_ILLEGAL;
>                  }
> 
> 		....
>          }

Excellent! Just what I was seeking!
> 
> but we must also mangle names ending in '.' and any
> names that match DOS reserved names of:
> 
> static const char * const reserved_names[] > { "AUX",
"LOCK$", "CON", "COM1", "COM2",
"COM3", "COM4",
>    "LPT1", "LPT2", "LPT3", "NUL",
"PRN"
> 
> Not that your mp3 files would match those :-).

I have been tipped off not to get NULL as my vanity license plate! ;-)

"How a 'NULL' License Plate Landed One Hacker in Ticket Hell"
https://www.wired.com/story/null-license-plate-landed-one-hacker-ticket-hell/


I am thankful,

-- 
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/