search for: io_flush

...ain.c *** main.c 1 Aug 2002 20:46:59 -0000 1.156 --- main.c 31 Dec 2002 01:21:34 -0000 *************** *** 346,351 **** --- 346,353 ---- exit_cleanup(0); } + io_start_buffering_in(f_in); + io_start_buffering_out(f_out); send_files(flist,f_out,f_in); io_flush(); report(f_out); *************** *** 421,427 **** close(error_pipe[1]); if (f_in != f_out) close(f_in); ! io_start_buffering(f_out); io_set_error_fd(error_pipe[0]); --- 423,429 ---- close(error_pipe[1]); if (f_in != f_out) close(f_in); ! io_start_buffering...

...ection, since a lot less data is being sent. Note also that without -I rsync is already very fast, since it skips (most) files based on attributes. With or without this patch the test suite passes except for daemon-gzip-upload. One risk of buffering is the potential for a bug caused by a missing io_flush: deadlock is possible, so try the patch at your own risk... Craig ########################################################################### diff -bur rsync/fileio.c rsync-craig/fileio.c --- rsync/fileio.c Fri Jan 25 15:07:34 2002 +++ rsync-craig/fileio.c Sat Dec 7 22:21:10 2002 @@ -76,7 +76,35...

..._links; extern int delete_after; extern int recurse; *************** *** 416,426 **** --- 417,435 ---- exit_cleanup(RERR_SOCKETIO); } + if (pipe(cleanup_pipe) < 0) { + rprintf(FERROR,"cleanup pipe failed in do_recv\n"); + exit_cleanup(RERR_SOCKETIO); + } + io_flush(); if ((pid=do_fork()) == 0) { + char tmp; + close(recv_pipe[0]); close(error_pipe[0]); + close(cleanup_pipe[1]); if (f_in != f_out) close(f_out); /* we can't let two processes write to the socket at one time */ *************** *** 436,450 **** write_int(recv_p...

...ut_of_memory("read_unbuffered"); > bufferSz = remaining; > } > read_loop(fd, buffer, remaining); > bufferIdx = 0; 309a331 > } 330c352,354 < --- > if (remaining == 0) { > io_flush(); > } 347,348d370 < io_flush(); < 534c556 < void io_start_buffering(int fd) --- > void io_start_buffering_out(int fd) 542a565,569 > void io_start_buffering_in(int fd) > { > multiplex_in_fd = fd; > } > 729c756 < io_start_buffering(fd); --- >...

...mp; (recurse || list_only)) { argc = 1; argv--; argv[0] = "."; } flist = send_file_list(f_out,argc,argv); if (!flist || flist->count == 0) { exit_cleanup(0); } the_file_list = flist; io_start_buffering_in(); io_start_buffering_out(); send_files(flist,f_out,f_in); io_flush(FULL_FLUSH); handle_stats(f_out); if (protocol_version >= 24) read_final_goodbye(f_in, f_out); io_flush(FULL_FLUSH); exit_cleanup(0); } **************************snip********************************** ************************************************************************** The prece...

Hello, I like to use rsync to transfer over mine and other peoples ever changing mail folder onto a remote server. My Server (FreeBSD 5.4) RSYNC Version is: 2.6.6 Protocol Version 29 My Client (OSX 10.4.2) RSYNC Version is: 2.6.3 Protocol Version 28 I use this command rsync -vvv --progress --stats --recursive --times \ --perms --links \ --delete /test/my?email?folder/*

...c to do it instead of doing it itself). Finally, the flushing code was tweaked to allow it to signal that the writing out of the receiver->generator pipe should be completely flushed (from the in-memory list of messages). This allows the final flushing to complete more efficiently than calling io_flush() in a loop. In my exuberance for having finally solved the redo-hang problem in a much better way than my previous suggested fixes for it, I went ahead and checked in my changes. However, feedback is still gratefully accepted. The changes can be found here: http://www.blorf.net/redo.patch...

FYI, We've been using the 2.4.7pre1 release for several days now, with nary a hang problem. We haven't seen the EOF bug at all, which was what we upgraded for. This is with transfers of as much as 50GB to set up an initial mirror. The only thing we did was set timeout=0 -- which I guess is unnecessary. The semantics of this flag are a bit unclear. We thought was 'time since

...tion has * started by looking e.g. at whether the remote version is known yet. */ -static int read_timeout (int fd, char *buf, int len) +static int read_timeout (int fd, void *buf1, int len) { int n, ret=0; + unsigned char * buf; + buf = (unsigned char *)buf1; io_flush(); @@ -236,8 +239,10 @@ /*! Continue trying to read len bytes - don't return until len has been read. */ -static void read_loop (int fd, char *buf, int len) +static void read_loop (int fd, void *buf1, int len) { + unsigned char * buf; + buf = (unsigned char *)buf1;...

I compiled rsync-2.5.5 on HPUX 11.11, using the +DA2.0W and +O3 options. invoking a simple rsync to transfer a file works (I ran a diff on the file, no changes) e.g: sdx1 214: ./rsync --rsh='/usr/bin/ssh -x' --rsync-path=/usr/local/src/rsync-2.5.5/rsync /scratch/chuck/tmp.test sdx2:/scratch/chuck However, adding the -a option yields an unexplained error: (In all of the following cases

...static int no_flush; * give a better explanation. We can tell whether the connection has * started by looking e.g. at whether the remote version is known yet. */ -static int read_timeout(int fd, char *buf, int len) +static int read_timeout (int fd, char *buf, int len) { int n, ret=0; io_flush(); while (ret == 0) { + /* until we manage to read *something* */ fd_set fds; struct timeval tv; int fd_count = fd+1; @@ -165,41 +210,27 @@ static int read_timeout(int fd, char *bu if (io_timeout) last_io = time(NULL); continue; + } else if (n == 0) { + whine_about_eof...

...g -1. The attached patch does 1 and 2 and works for our transfers. -------------- next part -------------- *** main.c.orig Tue Mar 26 23:10:44 2002 --- main.c Mon Apr 8 13:37:27 2002 *************** *** 37,42 **** --- 37,43 ---- while (waitpid(pid, status, WNOHANG) == 0) { msleep(20); io_flush(); + *status = 0; } /* TODO: If the child exited on a signal, then log an *************** *** 846,852 **** signal(SIGUSR1, sigusr1_handler); signal(SIGUSR2, sigusr2_handler); ! signal(SIGCHLD, sigchld_handler); #ifdef MAINTAINER_MODE signal(SIGSEGV, rsync_pa...

Hi, I've wanted to start working with this CVS version cuz the new "incremental-recursion algorithm" is just what I need. But ran into a problem... When I start the rsync, either with the rsync protocol or rsh, i found that it'll start doing the rsync and just halt after a few hundred MBs or even up to a couple GBs. I was never able to finish an rsync. Anyone else had this

...- if (read_batch) - create_flist_from_batch(); /* sets batch_flist */ ret = local_child(argc, args, f_in, f_out, child_main); } else { ret = piped_child(args,f_in,f_out); --- 321,326 ---- *************** *** 461,466 **** --- 480,491 ---- recv_files(f_in,flist,local_name); io_flush(FULL_FLUSH); + + /* This test for read_batch is needed because of a + * protocol dependency on am_server state, see + * report(). We are quite fortunate that this + * workaround is not more complicated. */ + if (!read_batch) report(f_in); send_msg(MSG_DONE, "",...

To Whom It May Concern: Below is a patch, that I have used to eliminate the unexplained errors in the rsync program. I was able to trace the problem to the order in which the sigchld_handler and wait_process routines were executed. If sigchld_handler executes first it retrieves the status that wait_process needs to indicate proper rsync termination. The code below allows the sigchld_handler to

...trace on server: #0 0xff218224 in _poll () #1 0xff1cb808 in _select () #2 0x24bec in writefd_unbuffered (fd=1, buf=0xffbe5ed0 ">", len=66) at io.c:406 #3 0x24eac in mplex_write (fd=1, code=62, buf=0x591d8 "\a\020", len=62) at io.c:498 #4 0x24f24 in io_flush () at io.c:518 #5 0x24940 in readfd (fd=0, buffer=0xffbe7020 "?\002r\215?/\201R", N=4) at io.c:314 #6 0x24998 in read_int (f=0) at io.c:329 #7 0x199a4 in send_files (flist=0x574e8, f_out=1, f_in=0) at sender.c:110 #8 0x1d1e8 in do_server_sender (f_in=0, f_out=1, argc=1,...

Hi all, today we had a performance issue transfering a big amount of data where one file was over 50GB. Rsync was tunneled over SSH and we expected the data to be synced within hours. However after over 10 hours the data is still not synced ... The sending box has rsync running with 60-80 % CPU load (2GHz Pentium 4) while the receiver is nearly idle. So far I had no acces to the poblematic

Hi all. I once tried to rsync around 100 GB (10 million files), but version 2.6.6 needed too much RAM and was too slow. Is one of the snapshots stable enough to try this again? Greetings Sven -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url :

...both of which are called from client_run() in main.c after send_exclude_list(). Could you play with calling it before send_exclude_list()? You're sending the list from the receiver to the sender so you're in the second half client_run(), the do_recv() part. You may possibly need to call io_flush() more often, although I don't think so. Could that be the reason why io_start_buffering() wasn't turned on earlier? Looks like buffering can be disabled with io_end_buffering() if you need to. > Also I'm sure that the 1hr spent building the exclude list can be > greatly redu...

...in __kernel_vsyscall () #12 0x4748cddd in ___newselect_nocancel () from /lib/libc.so.6 #13 0x0806a397 in writefd_unbuffered (fd=0, buf=0xbfecc7bc "h\002", len=620) at io.c:1074 #14 0x080692e9 in mplex_write (code=Variable "code" is not available. ) at io.c:1169 #15 0x0806937b in io_flush (flush_it_all=-4) at io.c:1191 #16 0x0806943b in read_timeout (fd=8, buf=0xbfeccd7c "", len=4) at io.c:481 #17 0x080699d6 in read_loop (fd=8, buf=0xbfeccd7c "", len=4) at io.c:735 #18 0x08069a22 in read_msg_fd () at io.c:246 #19 0x08069e4d in get_redo_num (itemizing=1, code=FLOG...