On Mon, 10 Apr 2023 at 07:07, Peter Stuge <peter at stuge.se> wrote:> > Brian Candler wrote: > > > What's odd is that the length is *always* 1231976033 (which is > > > 0x496E7661 or "Inva" in ASCII).One thing that can cause this is if the libc writes to stderr (ie fd 2) on some classes of error. This is something libc should probably not do, since in things that are not simple command line tools (say, a ssh daemon) may be using fd 2 for something else entirely.> > Could you get a tcpdump when this happens? > > Or debug output from at least the client (run ssh with -vvv) or > preferably the server (run sshd with -ddd).That's probably not going to show it, but strace'ing either the client or the server will probably capture the error message in full. Since you're using 9.1, the message could be an "Invalid free", since there was a double-free bug in that release :-( -- Darren Tucker (dtucker at dtucker.net) GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860 37F4 9357 ECEF 11EA A6FA Good judgement comes with experience. Unfortunately, the experience usually comes from bad judgement.
> On Apr 10, 2023, at 7:24 AM, Darren Tucker <dtucker at dtucker.net> wrote: > > On Mon, 10 Apr 2023 at 07:07, Peter Stuge <peter at stuge.se> wrote: >> >> Brian Candler wrote: >>>> What's odd is that the length is *always* 1231976033 (which is >>>> 0x496E7661 or "Inva" in ASCII). > > One thing that can cause this is if the libc writes to stderr (ie fd > 2) on some classes of error. This is something libc should probably > not do, since in things that are not simple command line tools (say, a > ssh daemon) may be using fd 2 for something else entirely.Agreed. libc or libcrypto...> >>> Could you get a tcpdump when this happens? >> >> Or debug output from at least the client (run ssh with -vvv) or >> preferably the server (run sshd with -ddd). > > That's probably not going to show it, but strace'ing either the client > or the server will probably capture the error message in full. > > Since you're using 9.1, the message could be an "Invalid free", since > there was a double-free bug in that release :-( >Ah, good to know. One other thing I noticed: I have ssh issues to/from my Ubuntu VM or slow https downloads until I reboot it, then it works properly for a while until it doesn't. Wondering if it's also a possible device driver related issue? None of the other guests (OpenWrt, CentOS, Fedora, etc) seem to have network related issues. Just Ubuntu. And it happens when Ubuntu is either the client or the server (at least for ssh/scp). -Philip
> On Apr 10, 2023, at 7:24 AM, Darren Tucker <dtucker at dtucker.net> wrote: > > On Mon, 10 Apr 2023 at 07:07, Peter Stuge <peter at stuge.se> wrote: >> >> Brian Candler wrote: >>>> What's odd is that the length is *always* 1231976033 (which is >>>> 0x496E7661 or "Inva" in ASCII). > > One thing that can cause this is if the libc writes to stderr (ie fd > 2) on some classes of error. This is something libc should probably > not do, since in things that are not simple command line tools (say, a > ssh daemon) may be using fd 2 for something else entirely. > >>> Could you get a tcpdump when this happens? >> >> Or debug output from at least the client (run ssh with -vvv) or >> preferably the server (run sshd with -ddd). > > That's probably not going to show it, but strace'ing either the client > or the server will probably capture the error message in full. > > Since you're using 9.1, the message could be an "Invalid free", since > there was a double-free bug in that release :-( >Forgot to ask: does this bug manifest at any particular time, or just connection initiation? Because I can see it happen on a connection that's been up for days... either idle or experiencing heavy traffic... etc. -Philip