rsync - Jun 2003 - crash on 2 gig file

Hi, I'm still waiting for my list subscription but if I don't send in
this
bug report now I won't for who-knows-how-long and I want to get it in....

I'm using 2.5.6 compiled from
cvs on SCO Open Server 5.0.6
bothe machines are the same version of OS and the same copy of rsync.

on my live machine one of the database files eventually got to be 2 gigs
the file has since then been archived and reset to 0

I'm rsyncing  two different backup machines against the live machine every
night.

the 2 gig file grew to 2 gigs on the backup machines, but now, the rsync
job is crashing every night on this file on both machines and half the
files have not been getting updated for the last few months. We only
noticed it recently.

so to summarize the bug:
file on LIVE is 0 bytes
file on BACKUP is 2 gigs

on BACKUP you run:
  rsync -az --delete LIVE:/u/appl /u

causes all files in the list before the 2gig file to update, nothing else.

Of course I manually fixed the problem by deleteing the 2gig file from the
backup machines and so tonights rsync should re-create the 0-byte versions
and update the other files no problem, but I just wanted to log the bug.

Thanks

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani

Does SCO support files over 2GB in size?

 TMC

On Wed, Jun 11, 2003 at 04:16:38PM -0400, Brian K. White
wrote:
> Hi, I'm still waiting for my list subscription but if I don't send
in this
> bug report now I won't for who-knows-how-long and I want to get it
in....
> 
> I'm using 2.5.6 compiled from
> cvs on SCO Open Server 5.0.6
> bothe machines are the same version of OS and the same copy of rsync.
> 
> on my live machine one of the database files eventually got to be 2 gigs
> the file has since then been archived and reset to 0
> 
> I'm rsyncing  two different backup machines against the live machine
every
> night.
> 
> the 2 gig file grew to 2 gigs on the backup machines, but now, the rsync
> job is crashing every night on this file on both machines and half the
> files have not been getting updated for the last few months. We only
> noticed it recently.
> 
> so to summarize the bug:
> file on LIVE is 0 bytes
> file on BACKUP is 2 gigs
> 
> on BACKUP you run:
>   rsync -az --delete LIVE:/u/appl /u
> 
> causes all files in the list before the 2gig file to update, nothing else.
> 
> Of course I manually fixed the problem by deleteing the 2gig file from the
> backup machines and so tonights rsync should re-create the 0-byte versions
> and update the other files no problem, but I just wanted to log the bug.
> 
> Thanks
> 
> Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
>
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
> filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani
> 
> -- 
> To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-- 
Tomasz M. Ciolek	
*******************************************************************************
  email:  tmc at dreamcraft dot com dot au 
*******************************************************************************
	GPG Key ID: 0x41C4C2F0  Key available on www.pgp.net	
*******************************************************************************
  Everything falls under the law of change;	
  Like a dream, a phantom, a bubble, a shadow,
  like dew of flash of lightning.
  You should contemplate like this. 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.samba.org/archive/rsync/attachments/20030616/cbffec6c/attachment.bin

> On Wed, Jun 11, 2003 at 04:16:38PM -0400, Brian K. White wrote:
> > ...
> > 2.5.6 (protocol 26) from cvs on SCO Open Server 5.0.6
> > both machines are the same version of OS and the same copy of rsync.
> > ...
> > file on LIVE is 0 bytes
> > file on BACKUP is 2 gigs
> >
> > on BACKUP you run:
> >   rsync -az --delete LIVE:/u/appl /u
> >
> > causes all files in the list before the 2gig file to update, nothing
else.
> >
>
> Does SCO support files over 2GB in size?
>
> -- 
> Tomasz M. Ciolek
Nope.

Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani

Tomasz Ciolek wrote:
>  I assume you mean you have a single 2gb file, not 2gb worth worth of
>  files .
>  
>  
>  The 2gb barrier is probably your problem. Rsync will try to create a file
lists, and with a 2gb files in store you will make large RAM usage a norm.
> 
>  Now when a file is being updated, you might find that it temporarily
>  swells beyond 2gb and then goes back to the right size. If yuor OS
>  barfs at that time ... then you get a problem.
> 
>  Tomasz
> 
>  
> On Sun, Jun 15, 2003 at 09:51:13PM -0400, Brian K. White wrote:
> 
>>>On Wed, Jun 11, 2003 at 04:16:38PM -0400, Brian K. White wrote:
>>>
>>>>...
>>>>2.5.6 (protocol 26) from cvs on SCO Open Server 5.0.6
>>>>both machines are the same version of OS and the same copy of
rsync.
>>>>...
>>>>file on LIVE is 0 bytes
>>>>file on BACKUP is 2 gigs
>>>>
>>>>on BACKUP you run:
>>>>  rsync -az --delete LIVE:/u/appl /u
>>>>
>>>>causes all files in the list before the 2gig file to update,
nothing
>>
>>else.
>>
>>>Does SCO support files over 2GB in size?
>>>
>>>-- 
>>>Tomasz M. Ciolek
>>
>>Nope.
>>
>>Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
>>+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
>>filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani
>>
>>
>>
>>-- 
>>To unsubscribe or change options:
http://lists.samba.org/mailman/listinfo/rsync
>>Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
I can imagine that dealing with a file that reaches the OS's maximum file 
size limit is problematic. No surprize there. The point of the report was 
that, don't you think rsync should deal with this situation in a better 
way than crashing and thus failing to do the rest of the job it was in the 
middle of?

Call me ignorant, but it seems to me rsync should be able to deal with any 
file that the OS can deal with, including one that is exactly as large as 
it is possible for a file to get. I would not put the mere existance of 2 
gig files in the same "don't do that" category as say, not having
any free
space for temp, not having any swap, trying to deal with large file lists 
and large files without lots of ram etc...

-- 
Brian K. White  --  brian@aljex.com  --  http://www.aljex.com/bkw/
+++++[>+++[>+++++>+++++++<<-]<-]>>+.>.+++++.+++++++.-.[>+<---]>++.
filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani