Rsync 3.2.7 is running on the Gentoo computer, which doesn't have a version,
other than it's "current". I'm running the script from this
computer.
Rsync 3.1.2 is on the source computer, where the files come from, which is
Ubuntu 18.0.4.6.
I'm copying to a CIFS share mounted on the Gentoo computer.
The rsync scripts are all similar to this one:
/usr/bin/rsync -v -a --progress --exclude-from=${exclude} --safe-links
--itemize-changes --no-perms --no-owner --progress --stats \
alexa at labadmin-precision-tower-3620.montefiore.org:/home/alexa/
/mnt/data.einstein/luke/all_but_dat/alexa/desktop_bkup/profile
\>> /home/maurice/logs/rsync-client-alexa.log
I re-ran the scripts skipping this one. The next one was running and during that
period, ssh stopped responded to new connections, so it may be the case that the
failure is taking place across time, and it doesn't fail wholesale
immediately.
However, I have other scripts like these copying from other sources (not Ubuntu)
and they are not causing these failures.
On 6/3/23, 12:40 AM, "Perry Hutchison" <pluto at agora.rdrop.com
<mailto:pluto at agora.rdrop.com> <mailto:pluto at agora.rdrop.com
<mailto:pluto at agora.rdrop.com>>> wrote:
CAUTION: This email comes from an external source; the attachments and/or links
may compromise our secure environment. Do not open or click on suspicious
emails. Please click on the ?Phish Alert? button on the top right of the Outlook
dashboard to report any suspicious emails.
Maurice R Volaski via rsync <maurice.volaski at lists.samba.org
<mailto:maurice.volaski at lists.samba.org> <mailto:maurice.volaski at
lists.samba.org <mailto:maurice.volaski at lists.samba.org>>> wrote:
> I have an rsync script that it is copying one computer (over ssh)
> to a shared CIFS mount on Gentoo Linux, kernel 6.3.4. The script
> runs for a while and then at some point quits knocking my ssh
> session offline on all terminals and it blocks ssh from being able
> to connect again. Even restarting sshd doesn't help. Rsync has
> apparently killed it. I have to reboot.
For starters:
What OS and version is the rsync script running on?
Which end do you have to reboot? The machine running the script,
or the Gentoo Linux?
What versions of rsync are running on each end?
Can you show the command line that fails?
Based on the mention of multiple terminals, it sounds as if you
have a fairly complex ssh environment. Can you get it to fail in
a simpler environment, ideally with only one terminal?
Maurice R Volaski <maurice.volaski at einsteinmed.edu> wrote:> Rsync 3.2.7 is running on the Gentoo computer, which doesn't have > a version, other than it's "current". I'm running the script from > this computer. > > Rsync 3.1.2 is on the source computer, where the files come from, > which is Ubuntu 18.0.4.6. > > I'm copying to a CIFS share mounted on the Gentoo computer. > > The rsync scripts are all similar to this one: > > /usr/bin/rsync -v -a --progress --exclude-from=${exclude} --safe-links --itemize-changes --no-perms --no-owner --progress --stats \ > alexa at labadmin-precision-tower-3620.montefiore.org:/home/alexa/ /mnt/data.einstein/luke/all_but_dat/alexa/desktop_bkup/profile \ > >> /home/maurice/logs/rsync-client-alexa.log > > I re-ran the scripts skipping this one. The next one was running > and during that period, ssh stopped responded to new connections, > so it may be the case that the failure is taking place across > time, and it doesn't fail wholesale immediately. > > However, I have other scripts like these copying from other > sources (not Ubuntu) and they are not causing these failures.You have several moving parts, which complicates figuring out which of the various interactions is contributing to the problem. BTW anyone else on the list is more than welcome to weigh in. I am hardly an expert on rsync, and not at all familiar with the ins and outs of either Gentoo or CIFS. One thing which I think is most likely _not_ involved in the problem is sshd on the Gentoo system, and this is consistent with the observation that restarting sshd did not help. (If I'm reading the rsync command correctly, rsync on the Gentoo system is establishing the ssh connection and transferring the files over it. Gentoo's sshd would be involved only if the client were initiating the connection.) Is there anything interesting in the rsync logfile, especially near the end, or in the any of the involved machines' system logs (including the CIFS host) around the time of the hang? Does the Gentoo system have enough space for rsync to copy the files to a local drive, so that rsync and Samba are not both working on the same transfer at the same time? (The files can then be copied to the CIFS share in a separate step, using "cp -r" or some such.) If rsync still fails when arranged that way it would tend to eliminate CIFS as a factor (and it will simplify the environment); OTOH if that "solves" the problem you'll at least have a workaround. Totally separate from that, is this Ubuntu system the only client using Ubuntu 18.0.4.6 and/or rsync 3.1.2?
Nice job on converting each switch to it's equivalent human readable format! I used Gentoo for two decades or so. Now using Void Linux as I have little time for compiling. One item that might be noteworthy for those running Gentoo, or a compiled from source distribution, is including reporting the CFLAGS/LDFLAGS compiler options utilized for compiling/linking the code. Sometimes optimizations will cause self-compiled programs surface bugs not commonly seen, when otherwise using the more commonly used CFLAGS/LDFLAGS compiler options. When software gets flaky like this, eg. just disappearing or mysteriously quitting, usually is the main cause. Strace will usually catch the problem, while gdb/debugger might make things amazingly stable. Also, try a different file system, for ruling out the compiled (CIFS) kernel drivers. Some thoughts, but since you're already in the edu domain, likely already thought of all this already! Roger On 6/3/23, Maurice R Volaski via rsync <rsync at lists.samba.org> wrote:> Rsync 3.2.7 is running on the Gentoo computer, which doesn't have a version, > other than it's "current". I'm running the script from this computer. > > Rsync 3.1.2 is on the source computer, where the files come from, which is > Ubuntu 18.0.4.6. > > I'm copying to a CIFS share mounted on the Gentoo computer. > > The rsync scripts are all similar to this one: > > /usr/bin/rsync -v -a --progress --exclude-from=${exclude} --safe-links > --itemize-changes --no-perms --no-owner --progress --stats \ > alexa at labadmin-precision-tower-3620.montefiore.org:/home/alexa/ > /mnt/data.einstein/luke/all_but_dat/alexa/desktop_bkup/profile \ >>> /home/maurice/logs/rsync-client-alexa.log