search for: io_buffer_size

Hi, On systems with HAVE_SETATTRLIST, commit b936741 breaks compilation. This is because do_setattrlist_times hasn't been converted to STRUCT_STAT *stp. Here's a small patch to do it. Cheers, Filipe PS: I've been playing with IO_BUFFER_SIZE, MAX_MAP_SIZE and WRITE_SIZE. Any plans to make it configurable at runtime? It seems to make a big difference for large files on a fast link (about 5x in my case) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/2...

...dy Merrill Red Hat, Inc. -------------- next part -------------- diff -ur rsync-2.5.6/io.c rsync-2.5.6-sign/io.c --- rsync-2.5.6/io.c 2002-04-10 22:11:50.000000000 -0400 +++ rsync-2.5.6-sign/io.c 2003-06-05 14:05:54.000000000 -0400 @@ -605,7 +605,7 @@ } while (len) { - int n = MIN((int) len, IO_BUFFER_SIZE-io_buffer_count); + ssize_t n = MIN((ssize_t) len, IO_BUFFER_SIZE-io_buffer_count); if (n > 0) { memcpy(io_buffer+io_buffer_count, buf, n); buf += n; Only in rsync-2.5.6-sign/: io.c.orig diff -ur rsync-2.5.6/match.c rsync-2.5.6-sign/match.c --- rsync-2.5.6/match.c 2002-04-09 02:11:06...

...+453,7 @@ } if (ret <= 0) { - rprintf(FERROR,"erroring writing %d bytes - exiting\n", len); + rprintf(FERROR,"erroring writing %lu bytes - exiting\n", len); exit_cleanup(RERR_STREAMIO); } @@ -561,7 +561,7 @@ } while (len) { - int n = MIN(len, IO_BUFFER_SIZE-io_buffer_count); + int n = MIN((ssize_t)len, IO_BUFFER_SIZE-io_buffer_count); if (n > 0) { memcpy(io_buffer+io_buffer_count, buf, n); buf += n; diff -ru rsync-2.4.6/match.c rsync-2.4.6-fixed/match.c --- rsync-2.4.6/match.c Sat Jan 29 06:35:03 2000 +++ rsync-2.4.6-fixed/match.c Mon J...

...ned char *)buf1; err_list_push(); @@ -469,24 +480,26 @@ } -static char *io_buffer; +static unsigned char *io_buffer; static int io_buffer_count; void io_start_buffering(int fd) { if (io_buffer) return; multiplex_out_fd = fd; - io_buffer = (char *)malloc(IO_BUFFER_SIZE); + io_buffer = malloc(IO_BUFFER_SIZE); if (!io_buffer) out_of_memory("writefd"); io_buffer_count = 0; } /* write an message to a multiplexed stream. If this fails then rsync exits */ -static void mplex_write(int fd, enum logcode code, char *buf, int len)...

https://bugzilla.samba.org/show_bug.cgi?id=8512 Summary: rsync -a slower than cp -a Product: rsync Version: 3.1.0 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P5 Component: core AssignedTo: wayned at samba.org ReportedBy: linux.news at bucksch.org QAContact:

Hi, I'm having a problem rsyncing one file (since I signed it). It seems that the content of a file is able to cause problems in the protocol. building file list ... 28820 files to consider apt/packages/avifile/ apt/packages/avifile/avifile-0.7.34-1.dag.rh90.i386.rpm rsync: error writing 4 unbuffered bytes - exiting: Broken pipe rsync error: error in rsync protocol data stream (code

...size_t bufferIdx = 0; > static size_t bufferSz; 290c293 < if (!io_multiplexing_in || fd != multiplex_in_fd) --- > if (fd != multiplex_in_fd) 292a296,305 > if (!io_multiplexing_in && remaining == 0) { > if (!buffer) { > bufferSz = 2 * IO_BUFFER_SIZE; > buffer = malloc(bufferSz); > if (!buffer) out_of_memory("read_unbuffered"); > } > remaining = read_timeout(fd, buffer, bufferSz); > bufferIdx = 0; > } > 296c309,310 < read_loop(fd, buf...

Attached is a patch that makes window strides constant when files are walked with a constant block size. In these cases, it completely avoids all memmoves. In my simple local test of rsyncing 57MB of 10 local files, memmoved bytes went from 18MB to zero. I haven't tested this for a big variety of file cases. I think that this will always reduce the memmoves involved with walking a large

...char *buffer; + static size_t bufferIdx = 0; + static size_t bufferSz; - if (!io_multiplexing_in || fd != multiplex_in_fd) + if (fd != multiplex_in_fd) return read_timeout(fd, buf, len); + if (!io_multiplexing_in && remaining == 0) { + if (!buffer) { + bufferSz = 2 * IO_BUFFER_SIZE; + buffer = malloc(bufferSz); + if (!buffer) out_of_memory("read_unbuffered"); + } + remaining = read_timeout(fd, buffer, bufferSz); + bufferIdx = 0; + } + while (ret == 0) { if (remaining) { len = MIN(len, remaining); - read_loop(fd, buf, len); +...