search for: csum_length

...remote_version >= 27) + write_int(f, sum->s2length); + write_int(f, sum->remainder); } @@ -164,19 +172,18 @@ sum.count = (len + (block_len - 1)) / block_len; sum.remainder = (len % block_len); - sum.n = block_len; + sum.blength = block_len; sum.flength = len; + sum.s2length = csum_length; /* not needed here sum.sums = NULL; */ if (sum.count && verbose > 3) { rprintf(FINFO, "count=%ld rem=%ld n=%ld flength=%.0f\n", (long) sum.count, (long) sum.remainder, - (long) sum.n, (double) sum.flength); + (long) sum.blength, (double) sum.flength); } -...

...er versions: the onus is on @@ -385,7 +385,8 @@ OFF_T flength; /**< total file length */ size_t count; /**< how many chunks */ size_t remainder; /**< flength % block_length */ - size_t n; /**< block_length */ + size_t blength; /**< block_length */ + size_t s2length; /**< csum_length */ struct sum_buf *sums; /**< points to info for each chunk */ }; --- generator.c Sat Mar 22 03:38:21 2003 +++ generator.c.dynsumlen Sat Mar 22 09:47:43 2003 @@ -109,13 +109,21 @@ /* - send a header that says "we have no checksums" down the f_out fd + * NULL sum_struct me...

...atly inflate the patch. Also, this shouldn't prevent map_ptr() from being a wrapper to mmap() in the future, though the map_ptr() code will need to contain a fallback to read(). 3. Character devices and FIFO files can be read only once, so we can't "redo" them in the adaptive csum_length scheme (I think that it's safe to assume that block devices can be read multiple times, though). Thus such files must always be transferred with a generous csum_length, say, at least 8 bytes. I'm not sure how to handle this protocol-wise; as a temporary measure, I increased the initial...

I'm in the midst of coding a patch set for consideration that will bump the protocol version and have a couple of observations. The current minimum backwards-compatible protocol is 15 but we have code that checks for protocol versions as old as 12. If someone else doesn't beat me to it i'm considering cleaning out the pre-15 compatibility code. A backwards compatibility patch could

...e() +void close_batch_csums_file(void) { close(fdb); @@ -423,7 +424,7 @@ } } -int read_batch_csums_file(char *buff, int len) +int read_batch_csums_file(void *buff, int len) { static int fdb_open = 1; int bytes_read; @@ -469,7 +470,7 @@ extern int csum_length; - read_batch_csums_file((char *) &file_flist_entry, sizeof(int)); + read_batch_csums_file( &file_flist_entry, sizeof(int)); if (file_flist_entry != flist_entry) { rprintf(FINFO, "file_list_entry NE flist_entry\n"); rprintf(...

I'd like some opinions on a couple of long-standing rsync issues. My two oldest, uncommitted patches are: - A "no hang" patch that makes sure that the pipe from the receiver to the generator can't block with resend requests. - The "move files" patch that got changed into a --delete-sent-files option. For each item I have two questions -- do we need to

I'm having a problem with large files being rsync'd twice because of the checksum failing. The rsync appears to complete on the first pass, but then is done a second time (with second try successful). When some debug code was added to receiver.c, I saw that the checksum for the remote file & the temp file do not match on the first try, so (as expected) it repeats the rsync & the

Hi, I tried to compile rsync2.5.3 on Solaris5.7, and I still got the same errors as Solaris5.8, can you please help me to look at the problem? I got error messages: Configure was run successfully, then I ran make, I got: "batch.c", line 408: operands have incompatible types: pointer to unsigned long ":" pointer to unsigned int cc: acomp failed for batch.c ***

...be - doing and how many bytes there are in the last - chunk */ - write_int(f_out, s->count); - write_int(f_out, s->n); - write_int(f_out, s->remainder); - - for (i = 0; i < s->count; i++) { - write_int(f_out, s->sums[i].sum1); - write_buf(f_out, s->sums[i].sum2, csum_length); - } - } else { - /* we don't have checksums */ - write_int(f_out, 0); - write_int(f_out, block_size); - write_int(f_out, 0); - } + /* we don't have checksums */ + write_int(f_out, 0); + write_int(f_out, block_size); + write_int(f_out, 0); } @@ -162,57 +146,43 @@ generate ap...

...9 +2175,13 @@ void check_for_finished_files(int itemizing, enum logcode code, int check_redo) if (check_redo && (ndx = get_redo_num()) != -1) { OFF_T save_max_size = max_size; OFF_T save_min_size = min_size; + int save_max_age = max_age; + int save_min_age = min_age; csum_length = SUM_LENGTH; max_size = -1; min_size = -1; + max_age = 0; + min_age = 0; ignore_existing = -ignore_existing; ignore_non_existing = -ignore_non_existing; update_only = -update_only; @@ -2184,6 +2207,8 @@ void check_for_finished_files(int itemizing, enum logcode code, i...

...sizeof (int)); + write_batch_csums_file(s ? &s->count : &int_zero, sizeof(int)); if (s) { for (i = 0; i < s->count; i++) { @@ -417,8 +414,7 @@ && (i == s->count - 1)) { fdb_close = 1; } - write_batch_csums_file(s->sums[i].sum2, - csum_length); + write_batch_csums_file(s->sums[i].sum2, csum_length); } } } @@ -427,17 +423,18 @@ { static int fdb_open = 1; int bytes_read; + char filename[MAXPATHLEN]; if (fdb_open) { - - /* Set up file extension */ - strcat(rsync_csums_file, batch_file_ext); + /* Set up file extensi...

...nder.c:receive_sums() s->count = read_int(f); .. s->sums = (struct sum_buf *)malloc(sizeof(s->sums[0])*s->count); if (!s->sums) out_of_memory("receive_sums"); for (i=0; i < (int) s->count;i++) { s->sums[i].sum1 = read_int(f); read_buf(f,s->sums[i].sum2,csum_length); .. If I read this right, given high enough s->count makes the malloc() parameter wraps around to a few bytes while still reading data past it. Exploiting is probably pretty difficult if at all possible. There would have to be some interesting data after the returned pointer. I didn't...

...Two of the three have a very shallow structure, and they work fine. The third, however, has a fairly deep structure and many more files than the other two, and rsync'ing this directory causes a hang on the client side. Firing up gdb, I get: (gdb) list 486 phase++; 487 csum_length = SUM_LENGTH; 488 ignore_times=1; 489 490 if (verbose > 2) 491 rprintf(FINFO,"generate_files phase=%d\n",phase); 492 493 write_int(f,-1); 494 495 if (remote_version >= 13) { (gdb) n generate_files phase=1 493...

...can hint at a better match */ for (j++; j<s->count && targets[j].t == t; j++) { int i2 = targets[j].i; if (i2 == last_i + 1) { if (sum != s->sums[i2].sum1) break; if (memcmp(sum2,s->sums[i2].sum2,csum_length) != 0) break; /* we've found an adjacent match - the RLL coder will be happy */ i = i2; break; } } If you answer my question, I will thank to you.. If...

...you do the actual patching, so such a "blind" batch fileset may be faulty. In the above scenario that's not a serious problem -- after patching from the batch fileset on the CD-R, I'll run rsync in normal mode over the network to fix whatever's broken (or just rely on a large csum_length, once the MD4 hash checksum_seed issue I explained yesterday is addressed). Other plausible motivating scenarios: * Using a patch fileset for an incremental backup that lets you to revert to the base copy * Estimating the bandwidth/time an rsync operation will take before actually carrying it out...

I was startled to find that the --csum-length option is not implemented even though it is documented in the manpage. Not a good thing. Here's a patch that removes the manpage entry. I'd much rather not see this option and actually have the csum_length be per-file dynamic. Either way this patch should be applied at least until the code changes. The patch also brings rsync.1 and rsyncd.conf.5 up to date with their yodl source. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies...

I needed a tool to produce diffs of binary volumes, and to apply them as patches accordingly. So I tried the 'batch mode' that was added to rsync, but it invariably segfaulted on reading the batches... $ sh foo.rsync_argvs foo.rsync_argvs: line 1: 8971 Segmentation fault rsync -a -v --stats --read-batch=foo ${1:-FOO} rsync: error writing 64 unbuffered bytes - exiting: Broken pipe

...nly"; --- 1027,1036 ---- *** receiver.c Tue Oct 5 15:53:39 2004 --- /tmp/rsync-2.6.3/receiver.c Tue Sep 21 11:24:06 2004 *************** *** 22,32 **** extern int verbose; extern int recurse; extern int delete_after; extern int max_delete; - extern int delete_mdays; extern int csum_length; extern struct stats stats; extern int dry_run; extern int read_batch; extern int batch_gen_fd; --- 22,31 ---- *************** *** 119,147 **** for (i = local_file_list->count-1; i >= 0; i--) { if (max_delete && deletion_count > max_delete) break; if (!l...

There's a rather old bug report in Debian's bug tracking system (see http://bugs.debian.org/27126) about wanting to be able to specify the maximum file size, as well as the minimum file size. Here's the text: Sometimes, it's useful to specify a file size range one is interested in. For example, I'd like to keep an up-to-date mirror of Debian, but I currently

...ieving revision 1.1 diff -u -r1.1 batch.c --- batch.c 2002/03/21 21:31:52 1.1 +++ batch.c 2002/03/21 21:44:11 @@ -396,16 +396,17 @@ struct sum_struct *s) { size_t i; - unsigned int int_zero = 0; + unsigned int int_count; extern int csum_length; fdb_open = 1; /* Write csum info to batch file */ - /* FIXME: This will break if s->count is ever not exactly an int. */ + /* FIXME: This will break if s->count is ever greater than 2^32 + -1 */ write_batch_csums_file(flist_entry, sizeof(int)); -...