Howdy, We occassionally get the following error when running our nightly backups: rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) This happens more on one or two machines than on any of the others. We've looked high and low to see if we're mistakenly sending these signals, but nothing is that we can find. Does anyone know what this might be from? Is it the server or the client that's complaining about the signal? Thanks, David.
On Mon, Feb 04, 2002 at 12:24:02PM -0500, David Birnbaum wrote:> Howdy, > > We occassionally get the following error when running our nightly > backups: > > rsync error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > This happens more on one or two machines than on any of the others. We've > looked high and low to see if we're mistakenly sending these signals, but > nothing is that we can find. > > Does anyone know what this might be from? Is it the server or the client > that's complaining about the signal?The side of rsync that receives files forks into two processes, and one sends the other a SIGUSR1 when it experiences a problem. So this is probably a symptom of your real problem, not the real problem. - Dave Dykstra
When i was getting these, I traced the process and its children (solaris:
truss -f). I found that one of the spawned threads was experiencing an io
timeout while the filelist was building. I had set no timeout, but it did
it at 60 seconds every time. I found that this corresponded to a
SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY
setting my timeout to 86400 (1 day), i stopped those. Of course, then, it
choked farther along, but that's another story.
Try setting a timeout, even if you don't want one. Make it the longest
the process should ever take.
Tim Conway
tim.conway@philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
Dave Dykstra <dwd@bell-labs.com>
Sent by: rsync-admin@lists.samba.org
02/06/2002 10:16 AM
To: David Birnbaum <davidb@chelsea.net>
cc: rsync@lists.samba.org
(bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: Re: SIGUSR1 or SIGINT error
Classification:
On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum
wrote:> I suspected that might be the case...now...how to determine the
"real"
> problem? Does rsync log it somewhere? lsof shows that STDERR/STDOUT
are> going to /dev/null, so I hope it's not writing it there. Nothing
> informative in syslog, just the message about the SIG:
>
> Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] rsync
error: received SIGUSR1 or SIGINT (code 20) at
rsync.c(229)>
> Any clues?
I'm sorry, but I don't have any more suggestions.
- Dave Dykstra
Currently 2.5.1pre3. I haven't tested that problem lately, though.
I'll
get the newest up and try a full sync. It's worth a try. I'll feel
really stupid, though, if i've put all this work into newsync (perl
driving find|diff|tar|lzop) and it's fixed in rsync. I think our case
will always create problems, though, with the broken nfs unlink in the
nfs3 interface on the NAS, and the broken nfs2 client on the solaris
machines (mtime bug). I won't let this influence my test, though ;-).
Tim Conway
tim.conway@philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
Dave Dykstra <dwd@bell-labs.com>
Sent by: rsync-admin@lists.samba.org
02/06/2002 03:41 PM
To: Eric Whiting <ewhiting@amis.com>
cc: Tim Conway/LMT/SC/PHILIPS@AMEC
David Birnbaum <davidb@chelsea.net>
rsync@lists.samba.org
Subject: Re: SIGUSR1 or SIGINT error
Classification:
Looks like a fix for that went into 2.5.0. See revision 1.87 at
http://cvs.samba.org/cgi-bin/cvsweb/rsync/io.c
Tim & David, what version are you running?
2.5.2 has some serious problems, Eric. Try the latest development
snapshot at
rsync://rsync.samba.org/ftp/unpacked/rsync/
or
ftp://rsync.samba.org/pub/unpacked/rsync/
- Dave Dykstra
On Wed, Feb 06, 2002 at 11:33:43AM -0700, Eric Whiting
wrote:> Make that 2 of us who need to specify a large timeout.
>
> I have found that I have to set the timeout to a large value (10000) to
> get the rsyncs to run successfully. Leaving it at the default seemed to
> cause timeout/hang problems. Of course I still running a 2.4.6dev
> version. I had troubles with 2.5.[01]. (solaris/linux mix of of rsync
> clients/servers)
>
> I need to try 2.5.2 as soon as I get a chance. Looks like some good
> fixes are happening in 2.5.2.
>
> eric
>
>
>
> On Wed, 2002-02-06 at 10:39, tim.conway@philips.com wrote:
> > When i was getting these, I traced the process and its children
(solaris: > > truss -f). I found that one of the spawned threads was experiencing
an io > > timeout while the filelist was building. I had set no timeout, but it
did > > it at 60 seconds every time. I found that this corresponded to a
> > SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0. BY
> > setting my timeout to 86400 (1 day), i stopped those. Of course,
then, it > > choked farther along, but that's another story.
> > Try setting a timeout, even if you don't want one. Make it the
longest > > the process should ever take.
> >
> > Tim Conway
> > tim.conway@philips.com
> > 303.682.4917
> > Philips Semiconductor - Longmont TC
> > 1880 Industrial Circle, Suite D
> > Longmont, CO 80501
> > Available via SameTime Connect within Philips, n9hmg on AIM
> > perl -e 'print pack(nnnnnnnnnnnn,
> >
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
> > ".\n" '
> > "There are some who call me.... Tim?"
> >
> >
> >
> >
> > Dave Dykstra <dwd@bell-labs.com>
> > Sent by: rsync-admin@lists.samba.org
> > 02/06/2002 10:16 AM
> >
> >
> > To: David Birnbaum <davidb@chelsea.net>
> > cc: rsync@lists.samba.org
> > (bcc: Tim Conway/LMT/SC/PHILIPS)
> > Subject: Re: SIGUSR1 or SIGINT error
> > Classification:
> >
> >
> >
> > On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote:
> > > I suspected that might be the case...now...how to determine the
"real"> > > problem? Does rsync log it somewhere? lsof shows that
STDERR/STDOUT > > are
> > > going to /dev/null, so I hope it's not writing it there.
Nothing
> > > informative in syslog, just the message about the SIG:
> > >
> > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning]
rsync > > error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229)
> > >
> > > Any clues?
> >
> >
> > I'm sorry, but I don't have any more suggestions.
> >
> > - Dave Dykstra
> >
> >
> >
> >
> >
>
Well, it ran to completion this way, in about 7.5h, but i'm not certain i believe it. While i left --delete --force off (I have been horribly burned testing those on really big chunks before), I would expect that the destination would then end up with at least as much as the source. big and big1 are subdirectories of the same volume, and its only contents aside from very small directory containing between 10 and 20 Kb of scripts, so i don't see how the destination could end up 6-1/2Gb short. When my current operations complete, I'll try one with all the options turned on, and run the filesystem map generator from my project to see what differences it left. I have an idea of a mod to make the hard links check more efficient, but I don't understand C well enough. What i was thinking of was to keep the st_nlink part of the stat, and if it'snot a directory and nlink >1, save the path and inode in a seperate list. and leave them out of the main flist. That way, there's no processing of the items for which there's no possibility of a need to track hard links, then fix only one copy of each linked file, delete all the others, and link them back to it. I'm guessing that's a complete redo of the protocol, though. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Tools@lonnetsvr /users/Tools>cat doit #!/bin/sh /cadappl/encap/packages/rsync-cvs/bin/rsync --rsync-path=/cadappl/encap/packages/rsync-cvs/bin/rsync -WHav --stats --progress alta:/wan/lon-tools1/lon-tools1/big* /wan/lon-tools2/lon-tools2>doit.log 2>&1 </dev/null &Tools@lonnetsvr /users/Tools>grep ' ' doit.log receiving file list ... done big/tools/Tools/.microsoft/Favorites/Channels/Arcadia Bay Demo Channel/ big/tools/Tools/.microsoft/Favorites/Channels/The Microsoft Channel/ big1/cadappl1/hpux/iclibs/CMOS18/EXTERNALS/PcCMOS18sfliolib_nlm_ex/2.1/tools/adf/vital/sfliolib_nlm -> ../../vha/sfliolib_nlm big1/cadappl1/hpux/iclibs/CMOS18/EXTERNALS/PcCMOS18shliolib_nlm_ex/2.1/tools/adf/vital/shliolib_nlm -> ../../vha/shliolib_nlm big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18flviolib_spm/2.1.1/lib/flviolib_spm.src -> ../tools/vital/timing/flviolib_spm.src big1/cadappl1/hpux/iclibs/CMOS18/PcCMOS18flviolib_spm/2.1.1/tools/adf/vital/flviolib_spm -> ../../vha/flviolib_spm big1/cadappl1/hpux/latest -> /cadappl/perl/5.6.1 Number of files: 2727469 Number of files transferred: 0 Total file size: 114067347318 bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 68790028 Total bytes written: 16 Total bytes read: 68790044 wrote 16 bytes read 68790044 bytes 2531.79 bytes/sec total size is 114067347318 speedup is 1658.20 Tools@lonnetsvr /users/Tools>df -k /wan/lon-tools*/big/tools Filesystem kbytes used avail capacity Mounted on lon-tools1:big 150147795 121588653 28559142 81% /wan/lon-tools1/big lon-tools2:big 150147795 115027617 35120178 77% /wan/lon-tools2/big Tools@lonnetsvr /users/Tools> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Tim Conway tim.conway@philips.com 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(nnnnnnnnnnnn, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), ".\n" ' "There are some who call me.... Tim?" Dave Dykstra <dwd@bell-labs.com> Sent by: rsync-admin@lists.samba.org 02/07/2002 10:28 AM To: David Birnbaum <davidb@chelsea.net> cc: Tim Conway/LMT/SC/PHILIPS@AMEC Eric Whiting <ewhiting@amis.com> rsync@lists.samba.org Subject: Re: SIGUSR1 or SIGINT error Classification: The fix that went into 2.5.0 was for timeouts that were happening even when --timeout=0 (the default). Can any of you say for sure that it makes a difference with a new version when you go from --timeout=0 to a very large timeout? I want to see if Tim's experience with timeouts defaulting to 60 seconds is still happening, or if that was only something earlier. Of course, it's also entirely possible that the "SIGUSR1 or SIGINT error" message is being caused by a different problem. - Dave Dykstra On Thu, Feb 07, 2002 at 10:22:23AM -0500, David Birnbaum wrote:> I'm running 2.5.2. However, we had the same type of problem with 2.4.6, > which is what we were running before. If I had to guess, I would say > that we're seeing this error a little more often in 2.5.2. > > David. > > ----- > > On Thu, 7 Feb 2002 tim.conway@philips.com wrote: > > > Currently 2.5.1pre3. I haven't tested that problem lately, though.I'll> > get the newest up and try a full sync. It's worth a try. I'll feel > > really stupid, though, if i've put all this work into newsync (perl > > driving find|diff|tar|lzop) and it's fixed in rsync. I think our case > > will always create problems, though, with the broken nfs unlink in the > > nfs3 interface on the NAS, and the broken nfs2 client on the solaris > > machines (mtime bug). I won't let this influence my test, though ;-). > > > > Tim Conway > > tim.conway@philips.com > > 303.682.4917 > > Philips Semiconductor - Longmont TC > > 1880 Industrial Circle, Suite D > > Longmont, CO 80501 > > Available via SameTime Connect within Philips, n9hmg on AIM > > perl -e 'print pack(nnnnnnnnnnnn, > >19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),> > ".\n" ' > > "There are some who call me.... Tim?" > > > > > > > > > > Dave Dykstra <dwd@bell-labs.com> > > Sent by: rsync-admin@lists.samba.org > > 02/06/2002 03:41 PM > > > > > > To: Eric Whiting <ewhiting@amis.com> > > cc: Tim Conway/LMT/SC/PHILIPS@AMEC > > David Birnbaum <davidb@chelsea.net> > > rsync@lists.samba.org > > Subject: Re: SIGUSR1 or SIGINT error > > Classification: > > > > > > > > Looks like a fix for that went into 2.5.0. See revision 1.87 at > > http://cvs.samba.org/cgi-bin/cvsweb/rsync/io.c > > > > Tim & David, what version are you running? > > > > 2.5.2 has some serious problems, Eric. Try the latest development > > snapshot at > > rsync://rsync.samba.org/ftp/unpacked/rsync/ > > or > > ftp://rsync.samba.org/pub/unpacked/rsync/ > > > > - Dave Dykstra > > > > > > On Wed, Feb 06, 2002 at 11:33:43AM -0700, Eric Whiting wrote: > > > Make that 2 of us who need to specify a large timeout. > > > > > > I have found that I have to set the timeout to a large value (10000)to> > > get the rsyncs to run successfully. Leaving it at the default seemedto> > > cause timeout/hang problems. Of course I still running a 2.4.6dev > > > version. I had troubles with 2.5.[01]. (solaris/linux mix of ofrsync> > > clients/servers) > > > > > > I need to try 2.5.2 as soon as I get a chance. Looks like some good > > > fixes are happening in 2.5.2. > > > > > > eric > > > > > > > > > > > > On Wed, 2002-02-06 at 10:39, tim.conway@philips.com wrote: > > > > When i was getting these, I traced the process and its children > > (solaris: > > > > truss -f). I found that one of the spawned threads wasexperiencing> > an io > > > > timeout while the filelist was building. I had set no timeout,but it> > did > > > > it at 60 seconds every time. I found that this corresponded to a > > > > SELECT_TIMEOUT parameter, which was set to 60 if IO_TIMEOUT was 0.BY> > > > > > setting my timeout to 86400 (1 day), i stopped those. Of course, > > then, it > > > > choked farther along, but that's another story. > > > > Try setting a timeout, even if you don't want one. Make it the > > longest > > > > the process should ever take. > > > > > > > > Tim Conway > > > > tim.conway@philips.com > > > > 303.682.4917 > > > > Philips Semiconductor - Longmont TC > > > > 1880 Industrial Circle, Suite D > > > > Longmont, CO 80501 > > > > Available via SameTime Connect within Philips, n9hmg on AIM > > > > perl -e 'print pack(nnnnnnnnnnnn, > > > > > >19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),> > > > ".\n" ' > > > > "There are some who call me.... Tim?" > > > > > > > > > > > > > > > > > > > > Dave Dykstra <dwd@bell-labs.com> > > > > Sent by: rsync-admin@lists.samba.org > > > > 02/06/2002 10:16 AM > > > > > > > > > > > > To: David Birnbaum <davidb@chelsea.net> > > > > cc: rsync@lists.samba.org > > > > (bcc: Tim Conway/LMT/SC/PHILIPS) > > > > Subject: Re: SIGUSR1 or SIGINT error > > > > Classification: > > > > > > > > > > > > > > > > On Tue, Feb 05, 2002 at 11:28:54AM -0500, David Birnbaum wrote: > > > > > I suspected that might be the case...now...how to determine the > > "real" > > > > > problem? Does rsync log it somewhere? lsof shows that > > STDERR/STDOUT > > > > are > > > > > going to /dev/null, so I hope it's not writing it there. Nothing > > > > > informative in syslog, just the message about the SIG: > > > > > > > > > > Feb 5 09:49:41 hite rsyncd[9279]: [ID 702911 daemon.warning] > > rsync > > > > error: received SIGUSR1 or SIGINT (code 20) at rsync.c(229) > > > > > > > > > > Any clues? > > > > > > > > > > > > I'm sorry, but I don't have any more suggestions. > > > > > > > > - Dave Dykstra > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[adding the mailing list to the Cc after David sent me trusses and other info about his hang. I have trimmed down the tracing info to the relevant pieces.] On Tue, Feb 19, 2002 at 11:51:27PM -0500, David Birnbaum wrote:> I wasn't able to get a netstat this time. However, I should be able to > grab it again if these truss's don't help. I'm thinking I will run a > snoop the next time and capture the packet traffic to see what's happening > at the TCP level - at least we should be able to see who/what shut down > first, because from the truss, it looks like the kernel shut down the > whole thing at once. It's tough to time the netstat because it takes a > while for it to die.I think the trusses with timestamps showed me enough.> Trusses, lsof, netstats attached. Thanks for the help; I appreciate the > efforts.The client is doing nothing complex, it's just merrilly blasting away down file descriptor 4 and never reading it. It then hangs and the only reason it wakes up is because the connection has been torn down. The server process looks very odd. It is doing a 196K read from a file, and it takes 9 minutes to return from that, and when it wakes up from that it finds that an EPIPE from the connection to the rsync client. Why would a 196K read take 9 minutes? It looks like it is going over NFS, so maybe the problem is with NFS. What's the network between the NFS server and the rsync server? Is it possible to run rsync on the NFS server itself, or is that a network appliance or something? It could be that the TCP implementation on the client side wasn't willing to wait for a response for 9 minutes and that's why it shut things down with an EPIPE. At the same time, the server child process is doing a 262K read from another file over the same NFS connection and it takes the same 9 minutes and eventually suceeds. It only dies because the other server process had died and it gets a signal.> David.- Dave Dykstra> Content-Description: client lsof > COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME...> rsync 20039 root 4u IPv4 0x3000dbd3810 0t64514407 TCP corona.chelsea.net:51816->hite.chelsea.net:rsync (ESTABLISHED) > rsync 20039 root 5r VREG 32,359 52273152 243313 /export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/cyberounds.com/view.dat> Content-Description: client truss...> 3504.6000 write(4, " w s 9 8 ; V Z _ I E".., 1021) = 1021 > 3504.6003 poll(0xFFBE5CA0, 1, 60000) = 1 > 3504.6005 write(4, "03FCFFFF", 4) = 4 > 3504.6010 poll(0xFFBE5CA0, 1, 60000) = 1 > 3504.6012 write(4, "02FCFFFF", 4) = 4 > 3504.6016 poll(0xFFBE5CA0, 1, 60000) = 1 > 3504.6019 write(4, "01FCFFFF", 4) = 4 > 3504.6034 poll(0xFFBE5CA0, 1, 60000) = 1 > 3504.6036 write(4, "E306\0\0", 4) = 4 > 3504.6041 poll(0xFFBE5D18, 1, 60000) = 1 > 3504.6046 write(4, " s t 1 0 3 . t n t 4 . r".., 1763) = 1763 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3564.6045 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3624.6045 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3684.6045 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3744.6046 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3804.6046 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3864.6044 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3924.6041 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 3984.6041 poll(0xFFBE5CA0, 1, 60000) = 0 > poll(0xFFBE5CA0, 1, 60000) (sleeping...) > 4037.3040 poll(0xFFBE5CA0, 1, 60000) = 1 > 4037.3046 write(4, "FEFBFFFF", 4) Err#32 EPIPE > 4037.3055 Received signal #13, SIGPIPE [caught] > 4037.3064 sigaction(SIGUSR1, 0xFFBE5838, 0xFFBE58B8) = 0 > 4037.3071 sigaction(SIGUSR2, 0xFFBE5838, 0xFFBE58B8) = 0 > 4037.3074 write(2, " r s y n c e r r o r :".., 66) = 66 > 4037.3084 mmap(0x00000000, 8192, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANON, -1, 0) = 0xFF310000 > 4037.3088 munmap(0xFF310000, 8192) = 0 > 4037.3091 llseek(0, 0, SEEK_CUR) = 583129 > 4037.3094 _exit(20)> Content-Description: server rsync lsof > COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME...> rsync 27948 root 5u IPv4 0x30000a59710 0t64477429 TCP hite.chelsea.net:rsync->corona.chelsea.net:51816 (ESTABLISHED) > rsync 27948 root 6uw VREG 85,30 0 541128 /export (/dev/md/dsk/d30) > rsync 27948 root 7r VREG 32,144 19062784 5347011 /export/backup -- export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/cyberounds.com/view.idx> Content-Description: server truss...> 3497.9255 lstat64("export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/resistance-forum-www.thebody.com/page.dat", 0xFFBEE370) = 0 > 3497.9262 open64("export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/resistance-forum-www.thebody.com/page.dat", O_RDONLY) = 7 > 3497.9869 read(7, "\002FFB7\0\0\0\0\002E0EF".., 196536) = 196536 > poll(0xFFBEC6E0, 2, 60000) (sleeping...) > 4037.2043 poll(0xFFBEC6E0, 2, 60000) = 0 > 4037.2049 poll(0xFFBEC6E0, 2, 60000) = 1 > 4037.2052 write(5, "FC0F\0078E hB40698 .84EF".., 4096) Err#32 EPIPE > 4037.2816 Received signal #13, SIGPIPE [caught] > 4037.2843 sigaction(SIGUSR1, 0xFFBEC2B0, 0xFFBEC330) = 0 > 4037.2848 sigaction(SIGUSR2, 0xFFBEC2B0, 0xFFBEC330) = 0 > 4037.2851 getpid() = 27948 [27130] > 4037.3504 kill(28192, SIGUSR1) = 0 > 4037.3508 getpid() = 27948 [27130] > 4037.3514 poll(0xFFBEABB0, 2, 60000) = 1 > 4037.3518 write(5, " B\0\0\b r s y n c e r".., 70) Err#32 EPIPE > 4037.3523 Received signal #13, SIGPIPE [default] > *** process killed ***> Content-Description: server child lsof > COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME...> rsync 28192 root 5u IPv4 0x30000a59710 0t64477429 TCP hite.chelsea.net:rsync->corona.chelsea.net:51816 (ESTABLISHED) > rsync 28192 root 6u VREG 85,30 0 541128 /export (/dev/md/dsk/d30) > rsync 28192 root 7r VREG 32,144 51363840 5347010 /export/backup -- export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/cyberounds.com/view.dat > rsync 28192 root 8u unix 105,36 0t0 306297 /devices/pseudo/tl@0:ticots->(socketpair: 0x18d400000000) (0x300016fd448) > rsync 28192 root 9u VREG 32,144 37569724 5347009 /export/backup -- export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/cyberounds.com/.view.dat.Q9aae3 > rsync 28192 root 10u unix 105,39 0t0 306297 /devices/pseudo/tl@0:ticots->(socketpair: 0x18d600000000) (0x300016fc368)> Content-Description: server child truss...> 3504.2667 poll(0xFFBECF20, 1, 60000) = 1 > 3504.2682 read(5, "D2\t\0\0", 4) = 4 > 3504.2699 poll(0xFFBECF98, 1, 60000) = 1 > 3504.2714 read(5, " I E 6 . 0 ; W i n d".., 2514) = 2514 > 3504.2738 write(9, " I E 6 . 0 ; W i n d".., 1024) = 1024 > 3504.2755 write(9, " . 0 ; W i n d o w s ".., 1024) = 1024 > 3504.2771 write(9, " w e r P C )\0 M9BCE9A 1".., 466) = 466 > 3504.2787 poll(0xFFBECF20, 1, 60000) = 1 > 3504.2800 read(5, "E4E6FFFF", 4) = 4 > 3504.2822 llseek(7, 5898240, SEEK_SET) = 5898240 > 3517.5204 read(7, "\0 Y \0\0\0\0\0\0 E16C5".., 262144) = 262144 > 4037.2227 write(9, " d o w s 9 5 )\0 T\v C".., 928) = 928 > 4037.2233 poll(0xFFBECF20, 1, 60000) = 1 > 4037.2238 read(5, "E3E6FFFF", 4) = 4 > 4037.2244 write(9, " 4 . 0 ( c o m p a t i".., 928) = 928 > 4037.2258 poll(0xFFBECF20, 1, 60000) = 1 > 4037.2262 read(5, "E2E6FFFF", 4) = 4 > 4037.2269 write(9, " c o m : M o z i l l a /".., 928) = 928 > 4037.2274 poll(0xFFBECF20, 1, 60000) = 1 > 4037.2278 read(5, "0E1A\0\0", 4) = 4 > 4037.2283 poll(0xFFBECF98, 1, 60000) = 1 > 4037.2288 read(5, " M o z i l l a / 4 . 0 ".., 6670) = 6670 > 4037.2304 write(9, " M o z i l l a / 4 . 0 ".., 1024) = 1024 > 4037.3011 write(9, " . c o m : M o z i l l a".., 1024) = 1024 > 4037.3017 write(9, " 2 4 7 - 2 9 - 0 . c l i".., 1024) = 1024 > 4037.3022 write(9, " 1 ; Q 3 1 2 4 6 1 )\0".., 1024) = 1024 > 4037.3028 write(9, " M S I E C r a w l e r )".., 1024) = 1024 > 4037.3079 llseek(9, 1, SEEK_CUR) = 585779 > 4037.3085 write(9, " WB7 lA0 1 2 - 2 5 5 - 1".., 1023) = 1023 > 4037.3125 write(9, " 0 7 2 1 2 0 0 0 h . 0 8".., 526) = 526 > 4037.3636 poll(0xFFBECF20, 1, 60000) = 1 > 4037.3642 read(5, " %DFFFFF", 4) = 4 > 4037.3650 llseek(7, 7733248, SEEK_SET) = 7733248 > 4037.6213 read(7, "\01A @\0\01A \0\0 6109E".., 262144) = 262144 > 4037.6229 Received signal #16, SIGUSR1 [caught] > siginfo: SIGUSR1 pid=27948 uid=0 > 4037.6234 sigaction(SIGUSR1, 0xFFBED188, 0xFFBED208) = 0 > 4037.6238 sigaction(SIGUSR2, 0xFFBED188, 0xFFBED208) = 0 > 4037.6913 unlink("export/httpd/DOMAINS/stats.chelsea.net/data/NetTracker/oral-forum-www.thebody.com/.visitorid.idx.Lkbae3") = 0 > 4037.6921 getpid() = 28192 [1] > 4037.6957 write(10, " A\0\0\b r s y n c e r".., 69) Err#32 EPIPE > 4037.6977 Received signal #13, SIGPIPE [caught] > 4037.6982 sigaction(SIGUSR1, 0xFFBEC8B0, 0xFFBEC930) = 0 > 4037.7016 sigaction(SIGUSR2, 0xFFBEC8B0, 0xFFBEC930) = 0 > 4037.7021 write(10, " A\0\0\b r s y n c e r".., 69) Err#32 EPIPE > 4037.7026 Received signal #13, SIGPIPE [default] > *** process killed ***