samba - Nov 2016 - slow directory access, convert_string_internal: Conversion error: Incomplete multibyte sequence

Mail file is server is a member server is Samba 3.6.26 on Solaris 11.    
It had been a BDC but a few week ago a switched it back to a member 
server.  The 2 current DC's had been running Samba 3.6.25 but switched 
to Samba 4.4.7.  This is  a classic domain.



Some Windows users reported that browsing folders shared from the member 
server have become very slow over the last few weeks. Which may 
correlate to either changing server role on the main file server or 
upgrading the DC's to Samba 4.


I do see this issue intermittently.  I open one file network folder with 
no problems.  I try another, and the progress bar progresses really 
slowly and it may time out.


Some client machine logs shows things like:


           smbd_dirptr_get_entry mask=[*] found
        somedirectory/somedirectory/somefile/@�!�P fname=@�!�P (@�!�P)
        [2016/11/29 18:23:02.279482,  3]
        lib/charcnv.c:537(convert_string_talloc)
           convert_string_talloc: Conversion error: Illegal multibyte
        sequence(�!�P)
        [2016/11/29 18:23:02.279560,  3]
        lib/charcnv.c:528(convert_string_talloc)
           convert_string_talloc: Conversion error: Incomplete multibyte
        sequence(�P)
        [2016/11/29 18:23:02.279628,  3]
        lib/charcnv.c:161(convert_string_internal)
           convert_string_internal: Conversion error: Illegal multibyte
        sequence(�!�P)
        [2016/11/29 18:23:02.279689,  3]
        lib/charcnv.c:140(convert_string_internal)
           convert_string_internal: Conversion error: Incomplete
        multibyte sequence(�P)
        [2016/11/29 18:23:02.279845,  3]
        smbd/dir.c:1049(smbd_dirptr_get_entry)



which may correlate to directories with files with invalid names


I don't seem to see entries correlating to directories I was trying to 
open .

testparm shows

         dos charset = CP850
         unix charset = UTF8


unix and samba data is all in LDAP so consistent idmapping is not a problem.


All shares have

         msdfs root = yes



This problem exists even on shared  volumes that seem to have low disk 
i/o load (using iostat.)



Appreciate any feedback or advice.


Thanks

On Tue, Nov 29, 2016 at 06:36:02PM -0500, Gaiseric Vandal via samba
wrote:
> Mail file is server is a member server is Samba 3.6.26 on Solaris
> 11.    It had been a BDC but a few week ago a switched it back to a
> member server.  The 2 current DC's had been running Samba 3.6.25 but
> switched to Samba 4.4.7.  This is  a classic domain.
> 
> 
> 
> Some Windows users reported that browsing folders shared from the
> member server have become very slow over the last few weeks. Which
> may correlate to either changing server role on the main file server
> or upgrading the DC's to Samba 4.
> 
> 
> I do see this issue intermittently.  I open one file network folder
> with no problems.  I try another, and the progress bar progresses
> really slowly and it may time out.
> 
> 
> Some client machine logs shows things like:
> 
> 
>           smbd_dirptr_get_entry mask=[*] found
>        somedirectory/somedirectory/somefile/@�!�P fname=@�!�P (@�!�P)
>        [2016/11/29 18:23:02.279482,  3]
>        lib/charcnv.c:537(convert_string_talloc)
>           convert_string_talloc: Conversion error: Illegal multibyte
>        sequence(�!�P)
>        [2016/11/29 18:23:02.279560,  3]
>        lib/charcnv.c:528(convert_string_talloc)
>           convert_string_talloc: Conversion error: Incomplete multibyte
>        sequence(�P)
>        [2016/11/29 18:23:02.279628,  3]
>        lib/charcnv.c:161(convert_string_internal)
>           convert_string_internal: Conversion error: Illegal multibyte
>        sequence(�!�P)
>        [2016/11/29 18:23:02.279689,  3]
>        lib/charcnv.c:140(convert_string_internal)
>           convert_string_internal: Conversion error: Incomplete
>        multibyte sequence(�P)
>        [2016/11/29 18:23:02.279845,  3]
>        smbd/dir.c:1049(smbd_dirptr_get_entry)
> 
> 
> 
> which may correlate to directories with files with invalid names
> 
> 
> I don't seem to see entries correlating to directories I was trying
> to open .
> 
> testparm shows
> 
>         dos charset = CP850
>         unix charset = UTF8
My guess is you have filenames on that share
that are not utf8. Look inside:

somedirectory/somedirectory/somefile/

There are definitely some files with some weird names-   in an ssh session they
don't even have regular characters.

e.g
-rw-rw----   1 xxx   xxx     114985112 Oct 31 14:39 ▒^t


Not sure if that is related to problems though.        

The top command shows

Memory: 12G phys mem, 343M free mem, 2048M total swap, 2048M free swap

This is in the evening so should not be much load but I think it was about the
same earlier.  Although I don't have a lot of free physical memory I am not
eating into my swap file.

Thanks


-----Original Message-----
From: Jeremy Allison [mailto:jra at samba.org] 
Sent: Tuesday, November 29, 2016 7:09 PM
To: gaiseric.vandal at gmail.com
Cc: Samba <samba at lists.samba.org>
Subject: Re: [Samba] slow directory access, convert_string_internal: Conversion
error: Incomplete multibyte sequence

On Tue, Nov 29, 2016 at 06:36:02PM -0500, Gaiseric Vandal via samba
wrote:
> Mail file is server is a member server is Samba 3.6.26 on Solaris
> 11.    It had been a BDC but a few week ago a switched it back to a
> member server.  The 2 current DC's had been running Samba 3.6.25 but 
> switched to Samba 4.4.7.  This is  a classic domain.
> 
> 
> 
> Some Windows users reported that browsing folders shared from the 
> member server have become very slow over the last few weeks. Which may 
> correlate to either changing server role on the main file server or 
> upgrading the DC's to Samba 4.
> 
> 
> I do see this issue intermittently.  I open one file network folder 
> with no problems.  I try another, and the progress bar progresses 
> really slowly and it may time out.
> 
> 
> Some client machine logs shows things like:
> 
> 
>           smbd_dirptr_get_entry mask=[*] found
>        somedirectory/somedirectory/somefile/@ ! P fname=@ ! P (@ ! P)
>        [2016/11/29 18:23:02.279482,  3]
>        lib/charcnv.c:537(convert_string_talloc)
>           convert_string_talloc: Conversion error: Illegal multibyte
>        sequence( ! P)
>        [2016/11/29 18:23:02.279560,  3]
>        lib/charcnv.c:528(convert_string_talloc)
>           convert_string_talloc: Conversion error: Incomplete multibyte
>        sequence( P)
>        [2016/11/29 18:23:02.279628,  3]
>        lib/charcnv.c:161(convert_string_internal)
>           convert_string_internal: Conversion error: Illegal multibyte
>        sequence( ! P)
>        [2016/11/29 18:23:02.279689,  3]
>        lib/charcnv.c:140(convert_string_internal)
>           convert_string_internal: Conversion error: Incomplete
>        multibyte sequence( P)
>        [2016/11/29 18:23:02.279845,  3]
>        smbd/dir.c:1049(smbd_dirptr_get_entry)
> 
> 
> 
> which may correlate to directories with files with invalid names
> 
> 
> I don't seem to see entries correlating to directories I was trying to 
> open .
> 
> testparm shows
> 
>         dos charset = CP850
>         unix charset = UTF8
My guess is you have filenames on that share that are not utf8. Look inside:

somedirectory/somedirectory/somefile/