similar to: rsync checksums

On Tue, Jan 28, 2003 at 11:22:14PM -0800, Craig Barratt wrote: > And I have several things I would like to work on and submit: > > - Fix the MD4 block and file checksums to comply with the rfc > (currently MD4 is wrong for blocks of size 64*n, or files > longer than 512MB). > > - Adaptive first pass checksum lengths: use 3 or more bytes of the MD4 > block

I apologize if this has already been discussed before, but as of yet I have been unable to find any info on the topic. I have a very simple (and common) disk based backup system using rsync, hard links, and a little bit of perl to glue it together. Remote machines are backed up regularly using hardlinks across each snapshot to reduce disk usage. Recently I learned that rsync does a

Hello Everyone, I've just started using rsync to copy files from Windows NT RCS library to Stratus VOS (Posix like fauilt tolerant Mini system) as a shadow. I would also like to setup rsync to copy log or other process output files from VOS to an NT system. Some questions if anyone here can help: 1. is there any computational or disk IO difference between the rsync client and server

Hi, while implementing the rsync protocol in one of our projects I found that the current CVS version still has a MD4 bug. I'm using the FreeBSD libmd implementation and I still had checksum mismatches with protocol version 27 for files whose size was a multiple of 64 - 4 ( - 4 due to checksum_seed). A patch for todays CVS version is attached. Someone should also review the clean_fname()

Back in the spring, we started using rsync for a disk to disk backup system maintaining close to 10PB of data. I am not here to debate the issue of what is the right tool but only to discuss what we found to be a problem with rsync when doing so. We traced the various processes hoping to find what the culprit was slowing things down so much and determined pretty easily that it was the

Wayne, I notice that .rsyncsums is starting to look a lot like the index that the git version control system ( http://git.or.cz/ ) uses to determine whether a file has changed since it was last staged for committing. The git index has been heavily used and tested, so you might find it helpful when implementing a checksum cache for rsync. Specifically, it has protection against being fooled when

This is a suggestion for adding to rsync an option "--checksum-path=PROGRAM". Once this option is given, provided it is accompained by option "--checksum", both the receiving and sending side of rsync would compute checksums calling PROGRAM, instead of the usual checksum program. Why is this usefull? Suppose I have two collections of MP3 files at different locations which I

Is anyone else concerned about the fact that rsync doesn't guarantee to produce identical file copies on the the target machine? Don't get me wrong in sounding critical because I think that rsync is a great example of how software should be written. (I often make the observation, as I learn more about Linux, and inevitably find myself comparing open source applications to Microsoft

I, too, was disappointed with rsync's performance when no changes were required (23 minutes to verify that a system of about 3200 files was identical). I wrote a little client/server python app which does the verification, and then hands rsync the list of files to update. This reduced the optimal case compare time to under 30 seconds. Here's what it does, and forgive me if these sound

Hi all! I've got a question about rsync using the MD5 algorithm. I know that rsync makes blocks of a file and then compares the checksums of each block. If the checksum is unequal the block is tranfered. Otherwise it could be that the checksum is the same but the block is different. Therefor rsync uses a 128 BIT MD5 algorithm. Now, at what filesize is it better to check the whole file

What is the best tool to compare file hashes in two different drives/directories such as after copying a large number of files from one drive to another? I used cp -au to copy directories, not rsync, since it is between local disks. I found a mention of hashdeep on the 'net which means first running it against the first directory generating a file with checksums and then running it a second

Thanks, rsync developers, for creating such a great tool. I'm a member of a team of system administrators and integrators that use rsync with SystemImager and Cfengine to rapidly deploy and prescriptively maintain systems. Our current practices would benefit if rsync were enhanced to handle duplicate names deterministically as described in the todo list. If we were able to use rsync to

Attached is a patch that implements compressing output files as they're written to disk, uzing zlib. Thus far I've only used it with synchronizing directories on a single machine. What seems to work / what's done: - Synchronizing directories with all files in the target directory gzip'd. Files seem to contain the correct data. Use the option "--gzip-dest". -

The attached patch provides an additional check for the checksumming mode to ensure that a file that is actually written out to disk can be read back and has the same MD4 sum as the file on at the originating location. Regards, Nick. -------------- next part -------------- *** rsync-2.6.6pre1/receiver.c 2005-04-14 02:42:13.000000000 +0100 --- rsync-new/receiver.c 2005-07-26

I'd like to know a little about the internals of RSYNC. I am a little confused as to why RSYNC is using both the simple 32 bit algorithm and the MD4 checksum function on the same files. From my testing this causes a vast overhead that is clearly not represented by RCP (fairly obvious!). Removing checksumming from a secure whole-file LAN transfer brings RSYNC in line with the general

On 02/09/2017 01:11 PM, Leonard den Ottolander wrote: > On Thu, 2017-02-09 at 12:58 -0600, Johnny Hughes wrote: >> At the time of extraction, the <name>.metadata file is created (again, >> not by us, but by the Red Hat team that distributes source), and all the >> non-text sha1sums are in there as well as all the text sources. > > Aha, <name>.metadata, well,

I am the author of BackupPC (http://backuppc.sourceforge.net) and I am working on adding rsync support to BackupPC. I am implementing the server-side in perl, and the client will run vanilla rsync. (BTW, is there the protocol documented? I've answered all my questions by looking at the source, but it would be great to check against any docs.) I started with librsync 0.9.3 and the

On 2020-05-22 22:54:18 [-0700], Wayne Davison via rsync wrote: > Thanks for the optimizing patches, Jorrit! I've merged your latest changes > into the git master branch. Wouldn't it be better to add support for a crypto library (like openssl) which would provide optimized algorithms for more than just one platform without the need to maintain it separately? > ..wayne..

Hi everyone! I played around with rsync sources a little and wrote a small patch that computes the checksums from parts of the files only. I'm just writing to ask if the rsync developers would have any interest in the sort of functionality described below. If you do, I'm willing to work with you to produce a cleaned up patch for git. For background: This started as a way to satisfy an

Hello John, On Thu, 2017-02-09 at 16:33 +0000, John Hodrien wrote: > On Thu, 9 Feb 2017, Leonard den Ottolander wrote: > > > How about my request for checksums in the git repo? > > What checksums would you actually want in git? SRPMS are signed which allows the integrity of the contents to be checked. Such an integrity check is missing from the git repo. Either a checksum