I am running rsync in cygwin on windows. I am attempting to backup a somewhat large data store (750GB) to a remote site. As its windows and preserving permissions exactly is important, I have an iSCSI drive mounted on the local system across a somewhat slow WAN link (IE, it would take about 3 months to copy the datastore over it). Unfortunately, since this appears as a "local" copy to rsync, it always copies whole files. Even though it is a "local" copy, I want to only send diffs, as we have large files that have small changes daily. Reading the man page, and everything I can find on the net I don't see an option to force diffs only/rsync protocol, is this possible? Thanks, Tom _________________________________________________________________ Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20100723/b3e6f871/attachment.html>
On Fri, 23 Jul 2010, Tom Christensen wrote:> I am running rsync in cygwin on windows. I am attempting to backup a > somewhat large data store (750GB) to a remote site. As its windows and > preserving permissions exactly is important, I have an iSCSI drive > mounted on the local system across a somewhat slow WAN link (IE, it > would take about 3 months to copy the datastore over it). > Unfortunately, since this appears as a "local" copy to rsync, it > always copies whole files. Even though it is a "local" copy, I want to > only send diffs, as we have large files that have small changes daily. > Reading the man page, and everything I can find on the net I don't see > an option to force diffs only/rsync protocol, is this possible?I didn't see a "don't use --whole-file"-type option. But the following doesn't use whole-file copies, as alluded to in the --whole-file documentation ("..., but only if no batch-writing option is in effect."): Create a file with the batched updates: rsync -a --only-write-batch=batched-updates local/disk/ iscsi/target/ Replay those updates: rsync -a --read-batch=batched-updates iscsi/target/ (or just:) ./batched-updates.sh Below is a test script that demonstrates. -- Best, Ben Steps: 1. Set up a test directory 2. Create a test file of 100MB of 0's, 12345, 100MB of 0's. 3. Sync it so there are two identical files. 4. Change the 12345 in the test file to 54321. 5. Generate the batch script. 6. Sync using the file. 7. Clean up. #!/bin/sh mkdir -p /tmp/rsync-test cd /tmp/rsync-test mkdir src perl -we 'print "0" x 100e6, 12345, "0" x 100e6' > src/file rsync -a src/ dest/ perl -we 'print "0" x 100e6, 54321, "0" x 100e6' > src/file rsync -a --only-write-batch=batched-updates src/ dest/ md5sum src/file dest/file ls -l batched-updates* ./batched-updates.sh md5sum src/file dest/file rm -rf /tmp/rsync-test
On 23.07.2010 00:30, Tom Christensen wrote:> > I am running rsync in cygwin on windows. I am attempting to backup a > somewhat large data store (750GB) to a remote site. As its windows > and preserving permissions exactly is important, I have an iSCSI drive > mounted on the local system across a somewhat slow WAN link (IE, it > would take about 3 months to copy the datastore over it). > Unfortunately, since this appears as a "local" copy to rsync, it > always copies whole files. Even though it is a "local" copy, I want > to only send diffs, as we have large files that have small changes > daily. Reading the man page, and everything I can find on the net I > don't see an option to force diffs only/rsync protocol, is this > possible?You have an abstraction error here, an iSCSI device is just a bit-bucket like any localy connected HDD in that all filesystem and data processing is done on the local side and raw block-data is send over the WAN. You would have to run the other side of rsync on the machine that provides the iSCSI-Device. For that to work the remote-machine would have to mount the filesystem localy which in most cases means you would have to unmount it from your Windows machine. This is because, except for cluster-filesystems or a filesystem that is mounted read-only (on every machine!), a given fileystem can only be mounted on 1 machine at a time. Bis denn -- Real Programmers consider "what you see is what you get" to be just as bad a concept in Text Editors as it is in women. No, the Real Programmer wants a "you asked for it, you got it" text editor -- complicated, cryptic, powerful, unforgiving, dangerous.