rsync - Dec 2003 - rsync'd destination much larger than source

Hello.

Recently, I started using rsync to backup files in my root partition on an 
Ensim
box over to a remote machine.  The remote machine 'pulls' from the Ensim
box
using the following:

rsync -arvzx --exclude=/proc --exclude=/tmp --exclude=/mnt --delete
--delete-excluded -e ssh 192.168.0.1:/  /bkup/rootpart/

The problem is, if I 'df' the Ensim box, it reports that the entire root
partition
is using 3.5 GB of space, however, the rsync'd copy of the root directory 
structure on the remote box is consuming over 23 GB.  Note that I did use -x 
as well as some excludes to ensure that files on other partitions mounted 
below '/ ' were not copied. Initially, my thoughts were that
symbolic links to directories were being treated as regular directories and
being copied repeatedly.  This does not appear to be the case, however.

The source server uses ext3, while the destination is ext2.  Blocksize is 1K 
in
both cases.

I'm certain I must be doing something wrong, but I don't know what.  Any
help is greatly appreciated.

TIA

Joe

_________________________________________________________________
Share holiday photos without swamping your Inbox.  Get MSN Extra Storage 
now!  http://join.msn.com/?PAGE=features/es

On Sun 30 Nov 2003, acct svcs wrote:
> 
> The problem is, if I 'df' the Ensim box, it reports that the entire
root
> partition
> is using 3.5 GB of space, however, the rsync'd copy of the root
directory
> structure on the remote box is consuming over 23 GB.  Note that I did use 
Perhaps there are sparse files on the source? Try using --sparse as
well.


Paul Slootman

Joe,

On Sun, 30 Nov 2003, acct svcs <heartofthematter@hotmail.com>
wrote:
> 
> rsync -arvzx --exclude=/proc --exclude=/tmp --exclude=/mnt --delete
> --delete-excluded -e ssh 192.168.0.1:/  /bkup/rootpart/
> 
> The problem is, if I 'df' the Ensim box, it reports that the entire
root
> partition
> is using 3.5 GB of space, however, the rsync'd copy of the root
directory
> structure on the remote box is consuming over 23 GB.  Note that I did use
-x
> as well as some excludes to ensure that files on other partitions mounted
> below '/ ' were not copied. Initially, my thoughts were that
> symbolic links to directories were being treated as regular directories and
> being copied repeatedly.  This does not appear to be the case, however.
Another factor is preserving hardlinks (-H option), but that won't explain
the large increase.

23 GB sounds about right to me if /var, /home, and /usr are not mount points
but are instead contained on the root partition (mine is 26 GB).  Can you
verify that the mount points are indeed only directories under /bkup/rootpart?

Also, are you sure you read the right column and used the right decimal place
on the df output?  Maybe the source has 3.5 GB _free_, not 3.5 GB used?
-- 
        John Van Essen  Univ of MN Alumnus  <vanes002@umn.edu>

Thank you for the suggestions.

Actually, I do want all of the root content (including /var, /usr, etc) to 
be included, and they are not separate mount points.  The source machine has 
a 132 GB drive.  Here's what df -h says:
Filesystem            Size  Used Avail Use% Mounted on
/dev/hda2             108G  3.3G   99G   4% /

I believe I may have found the problem, though.  The destination drive may 
have bad blocks, as I am getting Input/Output errors on some of the files 
that rsync created there.  I know the files are there (I can see them using 
find), but they can't be seen or manipulated with ls or cp.  They are not 
sparse files, either. I am speculating that this behavior may be causing 
rsync to act unpredictably because some of the files it writes can't be 
validated during the synchronization process.

Thanks

Joe

>From: John Van Essen <vanes002@umn.edu>
>To: <heartofthematter@hotmail.com>, <rsync@lists.samba.org>
>Subject: Re: rsync'd destination much larger than source
>Date: Sun, 30 Nov 2003 19:09:43 -0600
>MIME-Version: 1.0
>Received: from ms-smtp-01.rdc-kc.rr.com ([24.94.166.115]) by 
>mc5-f34.hotmail.com with Microsoft SMTPSVC(5.0.2195.6713); Sun, 30 Nov 2003 
>17:09:10 -0800
>Received: from jve-home (CPE-65-30-224-169.mn.rr.com [65.30.224.169])by 
>ms-smtp-01.rdc-kc.rr.com (8.12.10/8.12.7) with SMTP id hB1195oq026563;Sun, 
>30 Nov 2003 19:09:06 -0600 (CST)
>X-Message-Info: JGTYoYF78jHD72vniBuLhYXwvN8VpRFx
>X-Mailer: PocoMail 3.03 (1740) - Licensed Version
>Message-ID: <2003113019943.174346@jve-home>
>In-Reply-To: <BAY9-F54dH51ons00zY0003414f@hotmail.com>
>X-Virus-Scanned: Symantec AntiVirus Scan Engine
>Return-Path: vanes002@umn.edu
>X-OriginalArrivalTime: 01 Dec 2003 01:09:10.0521 (UTC) 
>FILETIME=[BA0AFE90:01C3B7A7]
>
>Joe,
>
>On Sun, 30 Nov 2003, acct svcs <heartofthematter@hotmail.com> wrote:
> >
> > rsync -arvzx --exclude=/proc --exclude=/tmp --exclude=/mnt --delete
> > --delete-excluded -e ssh 192.168.0.1:/  /bkup/rootpart/
> >
> > The problem is, if I 'df' the Ensim box, it reports that the
entire root
> > partition
> > is using 3.5 GB of space, however, the rsync'd copy of the root 
>directory
> > structure on the remote box is consuming over 23 GB.  Note that I did 
>use -x
> > as well as some excludes to ensure that files on other partitions 
>mounted
> > below '/ ' were not copied. Initially, my thoughts were that
> > symbolic links to directories were being treated as regular
directories
>and
> > being copied repeatedly.  This does not appear to be the case,
however.
>
>Another factor is preserving hardlinks (-H option), but that won't
explain
>the large increase.
>
>23 GB sounds about right to me if /var, /home, and /usr are not mount 
>points
>but are instead contained on the root partition (mine is 26 GB).  Can you
>verify that the mount points are indeed only directories under 
>/bkup/rootpart?
>
>Also, are you sure you read the right column and used the right decimal 
>place
>on the df output?  Maybe the source has 3.5 GB _free_, not 3.5 GB used?
>--
>         John Van Essen  Univ of MN Alumnus  <vanes002@umn.edu>
>
_________________________________________________________________
Groove on the latest from the hot new rock groups!  Get downloads, videos, 
and more here.  http://special.msn.com/entertainment/wiredformusic.armx