rsync - Oct 2003 - rsynced files "collecting" blocks?

Hello!

We are synching some server data (websites, logfiles, ...) with rsync via
ssh and are detecting a weird problem:

Certain files and directories, apparently those who are regulary rsynced over
the WAN, tend to use far too many blocks, e.g.:

stat from file on server:
File: `bilder.html'
Size: 2009            Blocks: 288        IO Block: 4096   regular file
[...]

The same file (a fresh copy) in normal state:
File: `bilder.html'
Size: 2039            Blocks: 8          IO Block: 4096   regular file

I am not entirely sure if it is a problem of rsync (2.5.6, linux i386 2.4.20+)
or perhaps of the file system (ext3, normal mode, with ACL support), but the
problem _exactly_ concerns to those files and directories that are synched via
WAN.

Synching happens nightly, in one direction (always the same), but sometimes we
abort (kill rsync) it for bandwidth reasons.

The problem does not concern other files including those that are regulary
synched via LAN.

Is this a known problem (did'nt find anything in the archive)?

The call is something like:
rsync /var/whatever foo.com:/var/whatever --timeout=180 --delete --force -g -l
-o -p -v -S -t --compress --progress --rsh ssh

Thanks in advance, and apologies if it should turn out as a ext3 problem
after all ;->>

kr

Christoph Lechleitner

------------------------------------------------------------------------
 IBCL - IT Bureau Dipl.-Ing. C. Lechleitner
 Defreggerstr. 24, A-6020 Innsbruck, Austria, Europe
 http://www.ibcl.net/
 Tel.: +43 512 390717, Fax: +43 512 390787, Mobile: +43 699 12353479
------------------------------------------------------------------------

On Tue, Oct 14, 2003 at 11:34:21PM +0200, Dipl.-Ing. C. Lechleitner
wrote:
> Hello!
> 
> We are synching some server data (websites, logfiles, ...) with rsync via
> ssh and are detecting a weird problem:
> 
> Certain files and directories, apparently those who are regulary rsynced
over
> the WAN, tend to use far too many blocks, e.g.:
> 
> stat from file on server:
Is this the sender or receiver?
> File: `bilder.html'
> Size: 2009            Blocks: 288        IO Block: 4096   regular file
> [...]
> 
> The same file (a fresh copy) in normal state:
And this?
> File: `bilder.html'
> Size: 2039            Blocks: 8          IO Block: 4096   regular file
> 
> I am not entirely sure if it is a problem of rsync (2.5.6, linux i386
2.4.20+)
> or perhaps of the file system (ext3, normal mode, with ACL support), but
the
> problem _exactly_ concerns to those files and directories that are synched
via
> WAN.
> 
> Synching happens nightly, in one direction (always the same), but sometimes
we
> abort (kill rsync) it for bandwidth reasons.
> 
> The problem does not concern other files including those that are regulary
> synched via LAN.
> 
> Is this a known problem (did'nt find anything in the archive)?
> 
> The call is something like:
> rsync /var/whatever foo.com:/var/whatever --timeout=180 --delete --force -g
-l -o -p -v -S -t --compress --progress --rsh ssh
> 
> Thanks in advance, and apologies if it should turn out as a ext3 problem
> after all ;->>
The fresh file looks ok.  The other one is insane.
> Size: 2009            Blocks: 288        IO Block: 4096   regular file
I assume that data is printed un-scaled from the stat(2)
structure.  If you have not somehow munged the fields you
have a corrupt filesystem.  There is no way under any
circumstances you could have a 2009 byte file using 288
blocks.  That is 144KB of allocated blocks, gross.

To clarify the meanings of these fields, size number of
bytes in the file (including NULs to fill in a sparse file),
blocks is the actual number of blocks allocated scaled to
512 size blocks.  The number of blocks will never be more
than the size rounded up to an even multiple of the block
size divided by 512.  A 2KB file on a 4KB filesystem will
use 1 filesystem block and have a reported block count of 8
(1 * 4096 / 512).

What you show can only happen if you have extracted the data
incorrectly or have a corrupt filesystem.  There is no way
that rsync could cause this.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt