thr3ads.net - samba - [Samba] Problem with large files corrupting during transfer [Aug 2006]

If this information is useful, please help other people find it:
Share via:

Ramsey Wes - wramse

2006-Aug-23 12:11 UTC

[Samba] Problem with large files corrupting during transfer

I'm very new to Samba, and I'm supporting my team's migration to a
new
server. We are running v3.0.23b, ML3 and AIX v5.3, and we're pulling
data from NT servers. V2.2.2 (what we are using on the old server) would
not compile on the new box, so we have to upgrade.

 

Files >2.8gb are corrupting during the transfer. That 2.8gb file is the
largest we've gotten to go through successfully. The next largest file
is 5.7gb, and corrupts without fail. The 1st section of the file looks
normal, the next large chunk is corrupt, and the remainder of the file
is simply binary trash. It looks to me like 2(+) users are accessing the
file while it is writing to cache, which would explain this, but I have
no way to prove it. I tried adding 'oplocks = no' and 'strict
locking yes', but neither fixed the problem. If more info is needed, please
let
me know.

 

Here's the smb.conf:

 

[global]

        workgroup = CORP

        server string = Qbert at Samba 3.0.23b

        interfaces = xxx.xx.xxx.xxx

        security = DOMAIN

        password server = *

        map to guest = Bad User

        log level = 2

        log file = /usr/local/samba/var/log.%m

        log file = /usr/local/samba/var/log.%S

        max log size = 10000

        deadtime = 150

        max open files = 2048

        local master = No

        domain master = No

        dns proxy = No

        wins server = 10.87.225.225

        create mask = 0774

        directory mask = 0775

        oplocks = No

        socket options = TCP_NODELAY

 

 

Here's the log.%S:

 

[2006/08/22 18:00:44, 2] smbd/reply.c:reply_special(496)

  netbios connect: name1=QBERT           name2=DPREP3

[2006/08/22 18:00:44, 2] smbd/reply.c:reply_special(503)

  netbios connect: local=qbert remote=dprep3, name type = 0

[2006/08/22 18:00:44, 2] smbd/sesssetup.c:setup_new_vc_session(795)

  setup_new_vc_session: New VC == 0, if NT4.x compatible we would close
all old resources.

[2006/08/22 18:00:44, 2] libsmb/namequery.c:name_query(577)

  Got a positive name query response from 10.87.225.225 ( xx.xx.xxx.xxx
xx.xx.xx.xxx xx.xx.xxx.xxx xx.xx.xxx.xxx xx.xx.xx.xxx...

[2006/08/22 18:00:44, 2] libsmb/namequery.c:name_query(577)

  Got a positive name query response from 10.87.225.225 ( 10.87.225.225
)

[2006/08/22 18:00:53, 1] lib/util_sock.c:open_socket_out(896)

  timeout connecting to 10.87.225.225:445

[2006/08/22 18:01:02, 1] lib/util_sock.c:open_socket_out(896)

  timeout connecting to 10.87.225.225:445

[2006/08/22 18:01:02, 2] auth/auth.c:check_ntlm_password(309)

  check_ntlm_password:  authentication for user [NAVSVR2] -> [NAVSVR2]
-> [navsvr2] succeeded

[2006/08/22 18:01:02, 2] smbd/reply.c:reply_tcon_and_X(711)

  Serving IPC$ as a Dfs root

[2006/08/22 18:01:02, 1] smbd/service.c:make_connection_snum(941)

  dprep3 (139.61.176.211) connect to service dpspot initially as user
navsvr2 (uid=4126, gid=209) (pid 283206)

[2006/08/22 18:01:02, 2] smbd/reply.c:reply_tcon_and_X(711)

  Serving dpspot as a Dfs root

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=Yes
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=No
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=No
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

[2006/08/22 18:21:16, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/per.file1 (numopen=0)

[2006/08/22 18:21:16, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/per.file1 read=No write=No (numopen=1)

[2006/08/22 18:21:16, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/per.cust1 (numopen=0)

[2006/08/22 18:31:54, 1] smbd/service.c:close_cnum(1141)

  dprep3 (139.61.176.211) closed connection to service dpspot

 

 

Here's the log.nmbd:

 

(This exact message is repeated every 5 minutes)

[2006/08/23 06:46:14, 2] nmbd/nmbd_elections.c:send_election_dgram(41)

  send_election_dgram: Sending election packet for workgroup CORP on
subnet xxx.xx.xxx.xxx

 

 

***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
****************************************************************************

Ramsey Wes - wramse

2006-Aug-24 13:47 UTC

head link

[Samba] RE: Problem with large files corrupting during transfer

List:

 

I've gotten more opinions on this problem, and the consensus is that
Samba is not compatible with Windows 2000 server (or NT) and our Unix
environment. If this is true I'm wasting my time. Is this a common or
known bug, or is it just something in the .conf files that needs to be
setup differently?

 

 

 

 

________________________________

From: Ramsey Wes - wramse 
Sent: Wednesday, August 23, 2006 7:01 AM
To: 'samba@lists.samba.org'
Subject: Problem with large files corrupting during transfer

 

I'm very new to Samba, and I'm supporting my team's migration to a
new
server. We are running v3.0.23b, ML3 and AIX v5.3, and we're pulling
data from NT servers. V2.2.2 (what we are using on the old server) would
not compile on the new box, so we have to upgrade.

 

Files >2.8gb are corrupting during the transfer. That 2.8gb file is the
largest we've gotten to go through successfully. The next largest file
is 5.7gb, and corrupts without fail. The 1st section of the file looks
normal, the next large chunk is corrupt, and the remainder of the file
is simply binary trash. It looks to me like 2(+) users are accessing the
file while it is writing to cache, which would explain this, but I have
no way to prove it. I tried adding 'oplocks = no' and 'strict
locking yes', but neither fixed the problem. If more info is needed, please
let
me know.

 

Here's the smb.conf:

 

[global]

        workgroup = CORP

        server string = Qbert at Samba 3.0.23b

        interfaces = xxx.xx.xxx.xxx

        security = DOMAIN

        password server = *

        map to guest = Bad User

        log level = 2

        log file = /usr/local/samba/var/log.%m

        log file = /usr/local/samba/var/log.%S

        max log size = 10000

        deadtime = 150

        max open files = 2048

        local master = No

        domain master = No

        dns proxy = No

        wins server = 10.87.225.225

        create mask = 0774

        directory mask = 0775

        oplocks = No

        socket options = TCP_NODELAY

 

 

Here's the log.%S:

 

[2006/08/22 18:00:44, 2] smbd/reply.c:reply_special(496)

  netbios connect: name1=QBERT           name2=DPREP3

[2006/08/22 18:00:44, 2] smbd/reply.c:reply_special(503)

  netbios connect: local=qbert remote=dprep3, name type = 0

[2006/08/22 18:00:44, 2] smbd/sesssetup.c:setup_new_vc_session(795)

  setup_new_vc_session: New VC == 0, if NT4.x compatible we would close
all old resources.

[2006/08/22 18:00:44, 2] libsmb/namequery.c:name_query(577)

  Got a positive name query response from 10.87.225.225 ( xx.xx.xxx.xxx
xx.xx.xx.xxx xx.xx.xxx.xxx xx.xx.xxx.xxx xx.xx.xx.xxx...

[2006/08/22 18:00:44, 2] libsmb/namequery.c:name_query(577)

  Got a positive name query response from 10.87.225.225 ( 10.87.225.225
)

[2006/08/22 18:00:53, 1] lib/util_sock.c:open_socket_out(896)

  timeout connecting to 10.87.225.225:445

[2006/08/22 18:01:02, 1] lib/util_sock.c:open_socket_out(896)

  timeout connecting to 10.87.225.225:445

[2006/08/22 18:01:02, 2] auth/auth.c:check_ntlm_password(309)

  check_ntlm_password:  authentication for user [NAVSVR2] -> [NAVSVR2]
-> [navsvr2] succeeded

[2006/08/22 18:01:02, 2] smbd/reply.c:reply_tcon_and_X(711)

  Serving IPC$ as a Dfs root

[2006/08/22 18:01:02, 1] smbd/service.c:make_connection_snum(941)

  dprep3 (139.61.176.211) connect to service dpspot initially as user
navsvr2 (uid=4126, gid=209) (pid 283206)

[2006/08/22 18:01:02, 2] smbd/reply.c:reply_tcon_and_X(711)

  Serving dpspot as a Dfs root

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=Yes
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=No
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/TEST92827.TXT read=No write=No
(numopen=1)

[2006/08/22 18:01:02, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/TEST92827.TXT (numopen=0)

[2006/08/22 18:01:02, 2] smbd/open.c:open_file(352)

[2006/08/22 18:21:16, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/per.file1 (numopen=0)

[2006/08/22 18:21:16, 2] smbd/open.c:open_file(352)

  NAVSVR2 opened file new.cust/per.file1 read=No write=No (numopen=1)

[2006/08/22 18:21:16, 2] smbd/close.c:close_normal_file(344)

  navsvr2 closed file new.cust/per.cust1 (numopen=0)

[2006/08/22 18:31:54, 1] smbd/service.c:close_cnum(1141)

  dprep3 (139.61.176.211) closed connection to service dpspot

 

 

Here's the log.nmbd:

 

(This exact message is repeated every 5 minutes)

[2006/08/23 06:46:14, 2] nmbd/nmbd_elections.c:send_election_dgram(41)

  send_election_dgram: Sending election packet for workgroup CORP on
subnet xxx.xx.xxx.xxx

 

 

***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
****************************************************************************

Ramsey Wes - wramse

2006-Aug-24 17:36 UTC

head link

[Samba] RE: Problem with large files corrupting during transfer

Jeremy,

We have an automated process that pulls data files from our server
(Qbert specifically) - by the way, ftp up is fine and files are moving
just fine. When the process/program is done manipulating the data it
writes it out from the NT or Win2k Server server directly to Qbert (so
that the data is not stored any place other than the destination server)
in 3 different file segments, using userid 'navsvr2'. The smaller two of
the 3 files are fine (<2gb), but the largest (5.7gb) gets corrupted.

If I can provide any other information, please let me know.

-----Original Message-----
From: Jeremy Allison [mailto:jra@samba.org] 
Sent: Thursday, August 24, 2006 10:53 AM
To: Ramsey Wes - wramse
Cc: samba@lists.samba.org
Subject: Re: [Samba] RE: Problem with large files corrupting during
transfer

On Thu, Aug 24, 2006 at 08:46:02AM -0500, Ramsey Wes - wramse
wrote:> 
> I've gotten more opinions on this problem, and the consensus is that
> Samba is not compatible with Windows 2000 server (or NT) and our Unix
> environment. If this is true I'm wasting my time. Is this a common or
> known bug, or is it just something in the .conf files that needs to be
> setup differently?
No, that's not true.
> server. We are running v3.0.23b, ML3 and AIX v5.3, and we're pulling
> data from NT servers. V2.2.2 (what we are using on the old server)
would> not compile on the new box, so we have to upgrade.
How are you pulling the data ? 

Jeremy.
***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
****************************************************************************

James Watkins

2006-Aug-25 08:27 UTC

head link

[Samba] Problem with large files corrupting during transfer

Ramsey Wes - wramse wrote:> Files >2.8gb are corrupting during the transfer. That 2.8gb file is the
> largest we've gotten to go through successfully. The next largest file
> is 5.7gb, and corrupts without fail. The 1st section of the file looks
> normal, the next large chunk is corrupt, and the remainder of the file
> is simply binary trash. It looks to me like 2(+) users are accessing the
> file while it is writing to cache, which would explain this, but I have
> no way to prove it. I tried adding 'oplocks = no' and 'strict
locking > yes', but neither fixed the problem. If more info is needed,
please let
> me know.
Hi, I'm afraid I can't help you but I can tell you that I had a similar 
problem that I gave up on some time ago.  I found that windows 2k could 
transfer the files perfectly well to my samba box so I guess it must be 
an NT problem.  I posted this to the list:

http://lists.samba.org/archive/samba/2006-May/120717.html

and only received one reply:

http://lists.samba.org/archive/samba/2006-May/120748.html

Having verified that I was running SP6, I lost interest and tried to 
persuade my boss to fund an upgrade to our windows servers instead.  I'm 
assured that the upgrade will happen some time this year :-) but I'd 
still be very interested to hear from you if you manage to fix the problem.

Good luck,

James.

Ramsey Wes - wramse

2006-Aug-25 12:58 UTC

head link

[Samba] Problem with large files corrupting during transfer

That's not exactly what I wanted to hear, but I guess we have no choice
but to build a work around. Our NT servers are no longer supported
because of an upgrade to a new system, and we're one of the few teams
left using the old system - therefore, I can't request any patches or
changes be applied on that end. Our migration has been discussed, but it
will take several months and I'm afraid we're already weeks behind going
to this new server.

One other option though...is there an older version of Samba that might
work between NT and our AIX box? We tried yesterday to install 2.2.8a,
and this is the return:

Compiling printing/nt_printing.c
Compiling smbd/noquotas.c
Compiling smbd/oplock.c
Compiling smbd/oplock_irix.c
Compiling smbd/oplock_linux.c
Compiling smbd/notify.c
Compiling smbd/notify_hash.c
Compiling smbd/notify_kernel.c
Linking bin/smbd
ld: 0711-317 ERROR: Undefined symbol: .VA_COPY
ld: 0711-317 ERROR: Undefined symbol: .SAFE_FREE
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
collect2: ld returned 8 exit status
make: 1254-004 The error code from the last command is 1.

Is it possible to fix the makefile so that it will work?

 
***************************************************************************
The information contained in this communication is confidential, is
intended only for the use of the recipient named above, and may be legally
privileged.

If the reader of this message is not the intended recipient, you are
hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited.

If you have received this communication in error, please resend this
communication to the sender and delete the original message or any copy
of it from your computer system.

Thank You.
****************************************************************************

Reasonably Related Threads

Search for more seemingly similar threads

samba - Aug 2006 - Problem with large files corrupting during transfer

[Samba] Problem with large files corrupting during transfer

[Samba] RE: Problem with large files corrupting during transfer

[Samba] RE: Problem with large files corrupting during transfer

[Samba] Problem with large files corrupting during transfer

[Samba] Problem with large files corrupting during transfer

Reasonably Related Threads