rsync - Nov 2004 - Database file rsync - any way to tell it to rsync against the last file?

My apologies if this has been asked before - I searched the archives
but couldn't find it, probably because I'm not sure what terms to use.

I'm trying to rsync database backups from point A to point B, where
point B has rsync running as a service.

Every day I make a new backup on point A (mike_db_20041110.bak), and
since there's no matching file on point B, the entire file is sent
over.  But since these are backups of the same database, the
difference between mike_db_20041110.bak and the prior backup,
(mike_db_20041109.bak), should be fairly minor.

Is there any way to tell it that I want to rsync mike_db_20041110.bak,
using mike_db_20041109.bak as a source, so that when I'm done I still
have both files?  I could build a script to rename them to the same
filename, but I'd rather let rsync do the work if possible.

Thanks for the help.  I've been using Rsync for our backups for a year
now, and absolutely love it.  Now that the backups are getting bigger,
it's time to get more aggressive.

Michael 

p.s. one other question - I know that Rsync compresses the file, but
does it look at the whole file first, then send?  The speeds I'm
getting sending these bak files doesn't look like it's being
compressed - the end file grows by about 10mb a minute, which is far
less than we should get with our T1.  And if it looks at the whole
file first, I should be seeing 10-to-1 compression.

Thanks.

Hi,

On Fri, 12 Nov 2004, Michael Bourgon wrote:
> My apologies if this has been asked before - I searched the archives
> but couldn't find it, probably because I'm not sure what terms to
use.
> 
> I'm trying to rsync database backups from point A to point B, where
> point B has rsync running as a service.
> 
> Every day I make a new backup on point A (mike_db_20041110.bak), and
> since there's no matching file on point B, the entire file is sent
> over.  But since these are backups of the same database, the
> difference between mike_db_20041110.bak and the prior backup,
> (mike_db_20041109.bak), should be fairly minor.
> 
> Is there any way to tell it that I want to rsync mike_db_20041110.bak,
> using mike_db_20041109.bak as a source, so that when I'm done I still
> have both files?  I could build a script to rename them to the same
> filename, but I'd rather let rsync do the work if possible.
Do a
  ln mike_db_20041109.bak mike_db_20041110.bak
before the rsync.
The time stamps should be different by nature, so rsync would communicate 
block checksums even if the sizes are equal, and transfer only the changed 
blocks
> p.s. one other question - I know that Rsync compresses the file, but
> does it look at the whole file first, then send?  The speeds I'm
> getting sending these bak files doesn't look like it's being
> compressed - the end file grows by about 10mb a minute, which is far
> less than we should get with our T1.  And if it looks at the whole
> file first, I should be seeing 10-to-1 compression.
Compression is a transfer option. No influence on file comparision.

Cheers -e
-- 
Eberhard Moenkeberg (emoenke@gwdg.de, em@kki.org)

On Fri, Nov 12, 2004 at 03:27:57PM -0600, Michael Bourgon
wrote:
> Is there any way to tell it that I want to rsync mike_db_20041110.bak,
> using mike_db_20041109.bak as a source, so that when I'm done I still
> have both files?
One solution that rsync supports out of the box is to use custom named
directories with the same file name.  For instance, change the files to
be these:

    20041109/mike_db.bak
    20041110/mike_db.bak

You could then run an rsync command like this:

  rsync -av --link-dest=/path/20041109 20041110/ remote:/path/20041110/

(The repeated 20041110 dir is necessary to make the lookup in the link-
dest dir work right.)

There is also a diff in the patches dir named fuzzy.diff that lets you
specify the --fuzzy option when you want rsync to try to find a matching
basis file in the current directory (it must have the same suffix and be
named similarly to the transferred file).  Using an rsync modified with
that patch would work with the filenames you're currently using.  I'm
considering adding --fuzzy for the next release, but I'd like to see it
improved a bit first.

..wayne..