search for: exit_cleanup

...ess on the other side may start to allocate all memory on the remote machine. As fa as we have analyzed the problem, the remote rsync process wants to issue a error message that it could not write 4096 bytes, receives a SIGPIPE and errno is set to EPIPE in the fwrite call (line 279 in log.c). Then exit_cleanup is called which in turn will call the same routine again and will of course get EPIPE again and the loop continues (as long as we have memory to recursivly call the same functions again and again). As a quick workaround we have the following patch, since there is no real point in trying to issue a...

...gets SIGPIPE. Alas, SIGPIPE is being ignored, because, quoting a comment in the rsync source code, "Ignore SIGPIPE; we consistently check error codes and will see the EPIPE." The comment is wrong; it does *not* see the SIGPIPE. What happens is that as a result of the SIGPIPE, exit_cleanup gets called. That's a macro which calls _exit_cleanup. That calls log_exit That calls rwrite. Rwrite tries to write an error to stderr, but that fails because of (you guessed it!) SIGPIPE, and so rwrite calls exit_cleanup. Presto, an infinite loop is born. The most straightforward...

...od_rsa" -vvv \\cagecfsu1.saifg.rbc.com\YCM0\InternalCache\test_aug17/ admin at 10.243.101.219:/cygdrive/g/EWCMS_SYNC/test_aug17/ 1>dump.txt 2>dump2.txt I get the following error in the dump.txt log file and nothing get synchronized. What sort of things should I be looking for? 1> exit_cleanup(code=12, file=io.c, line=609) 2> ssh_exchange_identification: Connection closed by remote host rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(609) [sender=3.0.6] Thanks in advanced -- View this messa...

...during a sync without the use of "-T", it will stop with this error: rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown": Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) That's because the code in receive_data() just calls exit_cleanup() upon a write error, which bombs out the receiver. For whatever reason, the sender doesn't correctly pick this up and it in turn aborts. People have "worked around" this problem by using "-T", putting the received file in temp space (assuming it's big enough). That a...

...size_only , 0, 0, 0 }, {"modify-window", 0, POPT_ARG_INT, &modify_window, OPT_MODIFY_WINDOW, 0, 0 }, {"one-file-system", 'x', POPT_ARG_NONE, &one_file_system , 0, 0, 0 }, @@ -471,6 +481,16 @@ print_rsync_version(FINFO); exit_cleanup(0); + case OPT_SOURCE_FILTER: + /* source_filter already set by popt */ + whole_file = 1; + break; + + case OPT_DEST_FILTER: + /* dest_filter already set by popt */ + whole_file = 1; + break; + case OPT_SUFFIX: /* The value has already been set by popt...

...code, two pairs are created as to_child_pipe and from_child_pipe ------------------------------------------------------------ --------------------- if (fd_pair(to_child_pipe) < 0 || fd_pair(from_child_pipe) < 0) { rsyserr(FERROR, errno, "pipe"); exit_cleanup(RERR_IPC); } //to_pipe_child[]={3,4} //from_pipe_child[]={5,6} pid = do_fork(); printf("value of pid is %d",pid); if (pid == -1) { rsyserr(FERROR, errno, "fork"); exit_cleanup(RERR_IPC);...

...quot;unknown": Broken pipe rsync error: error in rsync protocol data stream (code 12) at io.c(515) It ends up that if rsync gets a write error while transferring the file (such as is caused when you fill up a partition), it just fails. That's because the code in receive_data() just calls exit_cleanup() upon a write error, which bombs out the receiver. For whatever reason, the sender doesn't handle this gracefully, and in turn aborts. People have "worked around" this problem by using "-T", putting the received file in temp space (assuming it's big enough). That all...

...and transfer errors and always properly return with a non-zero exit code, despite many assurances to the contrary in the code and commit logs. It seems a kludge to handle a special case of lost connections to older servers was FAR too aggressive! With '-vvv' I also print the source of the exit_cleanup() call, and optionally with '-vvvv' show whether or not the exit code has been adjusted within _exit_cleanup(). A somewhat related minor style botch in main.c is fixed too. Also included here are my updated notes on possible improvements for handling 'moved files'. Index: cleanup...

...rgc, char *argv[]) { struct file_list *flist; char *dir = argv[0]; if (verbose > 2) { rprintf(FINFO, "server_sender starting pid=%ld\n", (long)getpid()); } if (am_daemon && lp_write_only(module_id)) { rprintf(FERROR, "ERROR: module is write only\n"); exit_cleanup(RERR_SYNTAX); return; } if (am_daemon && lp_read_only(module_id) && remove_sent_files) { rprintf(FERROR, "ERROR: --remove-sent-files cannot be used with a read-only module\n"); exit_cleanup(RERR_SYNTAX); return; } if (!relative_paths && !push_di...

...elow I think effects an equivalent synchronization, but does so by queuing a byte into a pipe rather than sending a signal. Of course, since it's not currently done this way, I may be overlooking something obvious. I can't quite see what though, since in the event that an error occurs then exit_cleanup is available to send SIGUSR1 with extreme prejudice; but if the protocol in fact concludes cleanly then there really should be no need for an asynchronous notification? Comments sought, meanwhile I'll test the patch a bit... Regards Anthony *** main.c.Orig Fri Jun 27 15:21:22 2003 --- main...

...e proto'. Eran -------------- next part -------------- diff -rupNP rsync-2.6.2/cleanup.c rsync-2.6.2-fnameconv.clean/cleanup.c --- rsync-2.6.2/cleanup.c 2004-01-27 10:14:33.000000000 +0200 +++ rsync-2.6.2-fnameconv.clean/cleanup.c 2004-05-29 21:17:08.000000000 +0300 @@ -119,6 +119,7 @@ void _exit_cleanup(int code, const char finish_transfer(cleanup_new_fname, fname, cleanup_file); } io_flush(FULL_FLUSH); + cleanup_fname_convert(); if (cleanup_fname) do_unlink(cleanup_fname); if (code) { diff -rupNP rsync-2.6.2/errcode.h rsync-2.6.2-fnameconv.clean/errcode.h --- rsync-2.6.2/errcode.h...

...e it if it doesn't exist + */ + fdb = do_open(filename, O_WRONLY | O_CREAT | O_TRUNC, S_IREAD | S_IWRITE); if (fdb == -1) { rprintf(FERROR, "Batch file %s open error: %s\n", - rsync_flist_file, strerror(errno)); + filename, strerror(errno)); close(fdb); exit_cleanup(1); } @@ -67,12 +50,11 @@ if (write(fdb, buff, bytes_to_write) == -1) { rprintf(FERROR, "Batch file %s write error: %s\n", - rsync_flist_file, strerror(errno)); + filename, strerror(errno)); close(fdb); exit_cleanup(1); } - if (fdb_close) { close(fdb); } @@...

...ut = MAX_DATA_COUNT; 284 r = deflate(&tx_strm, Z_INSERT_ONLY); 285 if (r != Z_OK || tx_strm.avail_in != 0) { 286 rprintf(FERROR, "deflate on token returned %d (%d bytes left)\n", 287 r, tx_strm.avail_in); 288 exit_cleanup(RERR_STREAMIO); 289 } MAX_DATA_COUNT is 16383 and toklen may be 16384. So, when deflate()'s outputs data length is bigger than input, then above code will fail. Normally output data length is smaller than input, but it is not true to already compressed data. --yasuoka Yasuoka Ma...

...- *************** *** 505,515 **** if (delete_mode && !delete_excluded) recv_exclude_list(f_in); ! ! if (read_batch) ! flist = batch_flist; ! else ! flist = recv_file_list(f_in); if (!flist) { rprintf(FERROR,"server_recv: recv_file_list error\n"); exit_cleanup(RERR_FILESELECT); --- 503,509 ---- if (delete_mode && !delete_excluded) recv_exclude_list(f_in); ! flist = recv_file_list(f_in); if (!flist) { rprintf(FERROR,"server_recv: recv_file_list error\n"); exit_cleanup(RERR_FILESELECT); *************** *** 602,607 **...

...; +#else + snprintf(err_buf, sizeof err_buf, + "Support for zstd is not enabled.\n"); + return 0; +#endif + break; + case 'M': arg = poptGetOptArg(pc); if (*arg != '-') { @@ -1856,7 +1877,12 @@ int parse_arguments(int *argc_p, const char ***argv_p) exit_cleanup(0); } - if (do_compression || def_compress_level != NOT_SPECIFIED) { + if (do_compression && do_compression_zstd) { + snprintf(err_buf, sizeof(err_buf), "Can't mix --compress and zstd\n"); + return 0; + } + + if (do_compression || (def_compress_level != NOT_SPECIFIED &am...

...m_daemon; + extern int module_id; + extern int am_sender; if (verbose > 2) rprintf(FINFO,"server_sender starting pid=%d\n",(int)getpid()); + if (am_daemon && lp_write_only(module_id) && am_sender) { + rprintf(FERROR,"ERROR: module is write only\n"); + exit_cleanup(RERR_SYNTAX); + return; + } + if (!relative_paths && !push_dir(dir, 0)) { rprintf(FERROR,"push_dir %s: %s (3)\n",dir,strerror(errno)); exit_cleanup(RERR_FILESELECT); diff -urN rsync-2.5.2.orig/proto.h rsync-2.5.2/proto.h --- rsync-2.5.2.orig/proto.h Sat Jan 26 00:07:33 20...

...=================================================================== RCS file: /cvsroot/rsync/main.c,v retrieving revision 1.156 diff -c -r1.156 main.c *** main.c 1 Aug 2002 20:46:59 -0000 1.156 --- main.c 31 Dec 2002 01:21:34 -0000 *************** *** 346,351 **** --- 346,353 ---- exit_cleanup(0); } + io_start_buffering_in(f_in); + io_start_buffering_out(f_out); send_files(flist,f_out,f_in); io_flush(); report(f_out); *************** *** 421,427 **** close(error_pipe[1]); if (f_in != f_out) close(f_in); ! io_start_buffering(f_out);...

> Just curious: what about the rsync+ patch? Thanks for the reminder. I've just committed Jos's rsync+ patch onto the "branch_mbp_rsyncplus_merge" branch. If it works OK and nobody screams I will move it across onto the main tree tomorrow or Wednesday. I see the patch doesn't add documentation about the new options to the man page, so we should fix that in the future.

...which is rsync <2.4.6 sending a list of modules on a + server, since the list is terminated by closing the socket. + So, for the section of the program where that is a problem + (start_socket_client), kludge_around_eof is True and we + just exit. + */ + + if (kludge_around_eof) + exit_cleanup (0); + else { + rprintf (FERROR, + "%s: connection unexpectedly closed " + "(%.0f bytes read so far)\n", + RSYNC_NAME, (double)stats.total_read); + + exit_cleanup (RERR_STREAMIO); + } +} -/* + +static void die_from_readerr (int err) +{ + /* this prevents us trying...

...c 2004-04-27 19:11:46.000000000 +0300 @@ -45,6 +45,7 @@ extern int cleanup_got_literal; extern int module_id; extern int ignore_errors; extern int orig_umask; +extern int do_fsync; static void delete_one(char *fn, int is_dir) { @@ -268,6 +269,12 @@ static int receive_data(int f_in,struct exit_cleanup(RERR_FILEIO); } + if (do_fsync && (fd != -1) && (fsync(fd) != 0)) { + rprintf(FERROR, "fsync failed on %s: %s\n", + full_fname(fname), strerror(errno)); + exit_cleanup(RERR_FILEIO); + } + sum_end(file_sum1); read_buf(f_in,file_sum2,MD4_SUM_LENGTH); @@ -458,7...