samba - Mar 2008 - Copying (Samba -> WinXP SP2) MATLAB generated .jpg files very slow.

Greetings,

This is a interesting problem. One of our researchers uses MATLAB to  
generate jpg files as part of his vision research. When he copies these  
files from the file server, it is very slow. I tried 3.0.28a and I see the  
same problem as we have with 3.0.28.

Here are some interesting stats.

1) MATLAB generated files:
command: robocopy Y:\23_1 C:\23_1 /e
dirs: 2
files: 262
bytes: 12.75MB
time: 3min 15 seconds


2) Same files resaved with IfranView
command: robocopy Y:\23_1b C:\23_1b /e
dirs: 2
files: 260
bytes: 22.05MB
time: 19 seconds

This is only a subset of the files that the user copies. The full  
directory consists of 8939 files over 101 directories for a total of 402MB  
and this takes 1 Hr,12 minutes and 5 seconds (Samba 3.0.28 -> WinXP SP2)  
verses 3 minutes 30 seconds for a WinXP SP2 -> WinXP SP2 copy.

When I write the files back to the same Samba server the results are about  
the same for the MATLAB and IfranView jpg files.

Any suggestions, since this only happens with MATLAB generated files?


Thanks

=====
/xsys/pkg/samba-3.0.28/sbin/smbd -b
Build environment:
    Built by:    paulg@navy
    Built on:    Tue Dec 11 09:46:16 EST 2007
    Built using: gcc
    Build host:  Linux navy 2.6.9-55.0.2.EL.CSE.smp #1 SMP Wed Jul 11  
09:50:28 EDT 2007 i686 i686 i386 GNU/Linux
    SRCDIR:      /tmp/xsys.8289.0/samba-3.0.28/source
    BUILDDIR:    /tmp/xsys.8289.0/samba-3.0.28/source

Paths:
    SBINDIR: /xsys/pkg/samba-3.0.28/sbin
    BINDIR: /xsys/pkg/samba-3.0.28/bin
    SWATDIR: /xsys/pkg/samba-3.0.28/swat
    CONFIGFILE: /xconf/samba/smb.conf
    LOGFILEBASE: /var/log
    LMHOSTSFILE: /xconf/samba/lmhosts
    LIBDIR: /xsys/pkg/samba-3.0.28/lib
    SHLIBEXT: so
    LOCKDIR: /var/samba/locks
    PIDDIR: /var/run
    SMB_PASSWD_FILE: /var/samba/private/smbpasswd
    PRIVATE_DIR: /var/samba/private
===
[global]
         workgroup = CSYORKUCA
         netbios name = PCSERVER
         netbios aliases = SILVER
         server string = CS Samba %v Server
         interfaces          bind interfaces only = Yes
         passdb backend = pdb_udb
         username map = /xconf/samba/usermap
         client NTLMv2 auth = Yes
         client lanman auth = No
         client plaintext auth = No
         log level = 2
         syslog = 0
         max log size = 0
         debug uid = Yes
         smb ports = 139
         deadtime = 15
         hostname lookups = Yes
         printcap name = /xconf/lprng/printcap
         logon script = default.bat
         logon path          logon home          domain logons = Yes
         os level = 65
         preferred master = Yes
         domain master = Yes
         wins support = Yes
         invalid users = root
         inherit permissions = Yes
         hosts allow          print command = /xsys/bin/lpr -b -P%p %s ; rm -f
%s
         lpq command = /xsys/bin/lpq -P%p
         lprm command = /xsys/bin/lprm -P%p %j
         use client driver = Yes
         oplocks = No
         level2 oplocks = No

[netlogon]
         path = /xconf/samba/netlogon

[homes]
         comment = Home Directories
         valid users = %S
         read only = No
         csc policy = disable

[printers]
         comment = All Printers
         path = /tmp
         create mask = 0700
         printable = Yes
         browseable = No

On Tue, Mar 11, 2008 at 12:08:36PM -0400, Paul Griffith
wrote:
>
> Greetings,
>
> This is a interesting problem. One of our researchers uses MATLAB to 
> generate jpg files as part of his vision research. When he copies these 
> files from the file server, it is very slow. I tried 3.0.28a and I see the 
> same problem as we have with 3.0.28.
>
> Here are some interesting stats.
>
> 1) MATLAB generated files:
> command: robocopy Y:\23_1 C:\23_1 /e
> dirs: 2
> files: 262
> bytes: 12.75MB
> time: 3min 15 seconds
>
>
> 2) Same files resaved with IfranView
> command: robocopy Y:\23_1b C:\23_1b /e
> dirs: 2
> files: 260
> bytes: 22.05MB
> time: 19 seconds
>
> This is only a subset of the files that the user copies. The full directory
> consists of 8939 files over 101 directories for a total of 402MB and this 
> takes 1 Hr,12 minutes and 5 seconds (Samba 3.0.28 -> WinXP SP2) verses 3
> minutes 30 seconds for a WinXP SP2 -> WinXP SP2 copy.
>
> When I write the files back to the same Samba server the results are about 
> the same for the MATLAB and IfranView jpg files.
>
> Any suggestions, since this only happens with MATLAB generated files?
Does MATLAB add any stream metadata to the files ?

robocopy may be doing something dumb with files with
associated streams.

Jeremy.