bugzilla-daemon at mindrot.org
2023-Dec-02 20:45 UTC
[Bug 3639] New: server thread aborts during client login after receiving SSH2_MSG_KEXINIT
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 Bug ID: 3639 Summary: server thread aborts during client login after receiving SSH2_MSG_KEXINIT Product: Portable OpenSSH Version: 9.2p1 Hardware: ARM OS: Linux Status: NEW Severity: critical Priority: P5 Component: sshd Assignee: unassigned-bugs at mindrot.org Reporter: jtm.moon.forum.user+mindrot at gmail.com tl;dr I downloaded and compiled openssh-9.2p1.tar.gz . When an openssh client attempts to login it sends SSH2_MSG_KEXINIT, the server immediately resets the connection ### Reproduction Steps Using Raspbian 11 (based on Debian 11 Bullseye) on a Raspberry Pi 4 (ARM aarch64), I downloaded https://mirror.edgecast.com/pub/OpenBSD/OpenSSH/portable/openssh-9.2p1.tar.gz I compiled and installed it. First, make sure necessary build packages are available apt install \ libssl-dev \ gcc g++ gdb cpp \ make cmake \ libtool \ libc6 \ autoconf automake pkg-config \ build-essential \ gettext \ libzstd1 zlib1g \ libssh-4 libssh-dev libssl3 \ libc6-dev libc6 \ libcrypt-dev Download, build, install cd /tmp wget https://mirror.edgecast.com/pub/OpenBSD/OpenSSH/portable/openssh-9.2p1.tar.gz tar -xvf openssh-9.2p1.tar.gz cd openssh-9.2p1 ./configure --prefix=/opt/openssh-9.2p1 make make install Adjust sshd_config vim /opt/openssh-9.2p1/etc/sshd_config Add lines for a unique port, 2232, increase the log level Port 2232 LogLevel DEBUG3 Otherwise, the `sshd_config` is used as-is. Manually start `sshd` /opt/openssh-9.2p1/sbin/sshd -D Tail the logs tail -f /var/log/auth.log On a different host, attempt to login using the openssh client PS> ssh.exe root at 192.168.1.2 -p 2232 -vvvv The tail of the output shows ... debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: recv - from CB ERROR:10054, io:000002E46F4CB690 Connection reset by 192.168.1.2 port 2232 That output is from Windows ssh.exe (OpenSSH_for_Windows_8.6p1, LibreSSL 3.4.3). Using Ubuntu 22 x64 ssh (OpenSSH_8.9p1 Ubuntu-3ubuntu0.4, OpenSSL 3.0.2 15 Mar 2022) the ssh client output looks like: $ ssh root at 192.168.1.2 -p 2232 -vvvv ... debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent Connection reset by 192.168.1.2 port 2232 Using Debian 11 ARM ssh client compiled from the same compilation (OpenSSH_9.2p1, OpenSSL 1.1.1w 11 Sep 2023) the same error occurs. $ /opt/openssh-9.2p1/bin/ssh -p 2232 root at 192.168.1.2 -vvvvv ... debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent Connection reset by 192.168.1.2 port 2232 The server log messages from `/var/log/auth.log` are 2023-12-02T12:28:41.051665-08:00 host1 sshd[3790]: Connection from 192.168.1.3 port 62155 on 192.168.1.2 port 2232 rdomain "" 2023-12-02T12:28:41.050817-08:00 host1 sshd[3790]: Connection from 192.168.1.3 port 62155 on 192.168.1.2 port 2232 rdomain "" 2023-12-02T12:28:41.053381-08:00 host1 audit[3791]: SECCOMP auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=3791 comm="sshd" exe="/opt/openssh-9.2p1/sbin/sshd" sig=31 arch=40000028 syscall=20 compat=1 ip=0xf787080c code=0x0 ### Notes This error does not occur using release 9.1p1. This error does occur for release 9.2p1 up to 9.5p1 (I tried all of them). I attempted to reproduce this on a Ubuntu 22 x64 Virtual Machine as the server. The error did not occur (logins succeeded). Various information about the host on which the error occurs $ lscpu Architecture: aarch64 Byte Order: Little Endian CPU(s): 4 On-line CPU(s) list: 0-3 Thread(s) per core: 1 Core(s) per socket: 4 Socket(s): 1 Vendor ID: ARM Model: 3 Model name: Cortex-A72 Stepping: r0p3 CPU max MHz: 1500.0000 CPU min MHz: 600.0000 BogoMIPS: 108.00 L1d cache: 128 KiB L1i cache: 192 KiB L2 cache: 1 MiB Vulnerability Itlb multihit: Not affected Vulnerability L1tf: Not affected Vulnerability Mds: Not affected Vulnerability Meltdown: Not affected Vulnerability Mmio stale data: Not affected Vulnerability Retbleed: Not affected Vulnerability Spec store bypass: Vulnerable Vulnerability Spectre v1: Mitigation; __user pointer sanitization Vulnerability Spectre v2: Vulnerable Vulnerability Srbds: Not affected Vulnerability Tsx async abort: Not affected Flags: fp asimd evtstrm crc32 cpuid $ uname -a Linux host1 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux The host `apt` packages (dpkg) are up-to-date. I suspect this bug is specific to this Debian-derived distribution and/or ARM architecture. -- You are receiving this mail because: You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-02 20:47 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 JM <jtm.moon.forum.user+mindrot at gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jtm.moon.forum.user+mindrot | |@gmail.com -- You are receiving this mail because: You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-02 21:55 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 Darren Tucker <dtucker at dtucker.net> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dtucker at dtucker.net --- Comment #1 from Darren Tucker <dtucker at dtucker.net> --- (In reply to JM from comment #0) [...]> 2023-12-02T12:28:41.053381-08:00 host1 audit[3791]: SECCOMP > auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=3791 > comm="sshd" exe="/opt/openssh-9.2p1/sbin/sshd" sig=31 arch=40000028 > syscall=20 compat=1 ip=0xf787080c code=0x0This looks like a seccomp sandbox violation. The first thing I'd suggest is to try 9.5p1, because there were a couple of bugs fixed in that specifically (bug#3512 and bug#3537). There was also a thing about RNG seeding, but that depended on an interaction between different kernel and libc versions. Failing that, I'd suggest building with -DSANDBOX_SECCOMP_FILTER_DEBUG to get additional debugging on what's failing (but note that this configuration is for debugging only and is not safe for production use) and see what's in sshd's log. FWIW I have an rpi4 with a very similar configuration to what you describe ("Linux 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux") but was not able to reproduce the failure. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-03 05:13 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #2 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- tl;dr `strace` reveals error `"\0\0\0c\0\0\0\1\0\0\0\0\0\0\0Wssh_sandbox_violation: unexpected system call (arch:0x40000028,syscall:20 @ 0xf7ba380c)"` in response to a very large `prctl` Linux function call. (skip to the end to see)> try 9.5p1,Per my prior comment, I have tried 9.5p1. I tried 9.1p1 to 9.5p1. Only 9.1p1 didn't have this bug.> I'd suggest building with -DSANDBOX_SECCOMP_FILTER_DEBUG to get additional debugging on what's failingI tried this with 9.5p1. Built with the flag `DSANDBOX_SECCOMP_FILTER_DEBUG` CFLAGS="-DDSANDBOX_SECCOMP_FILTER_DEBUG" ./configure --prefix=/opt/openssh-9.5p1 make make install I could see in the output from `make` echoing the `cc` commands the `-DSANDBOX_SECCOMP_FILTER_DEBUG` being passed, e.g. cc -DSANDBOX_SECCOMP_FILTER_DEBUG -pipe -Wno-error=format-truncation -Wall -Wpointer-arith -Wuninitialized -Wsign-compare -Wformat-security -Wsizeof-pointer-memaccess -Wno-pointer-sign -Wno-unused-result -Wimplicit-fallthrough -Wmisleading-indentation -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -ftrapv -fno-builtin-memset -fstack-protector-strong -fPIC -I. -I.. -I. -I./.. -D_XOPEN_SOURCE=600 -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_GNU_SOURCE -DHAVE_CONFIG_H -c bindresvport.c (so it looks like I passed `-DSANDBOX_SECCOMP_FILTER_DEBUG` correctly) I added to /opt/openssh-9.5p1/etc/sshd_config LogLevel DEBUG3 Then I ran `tail -f /var/log/auth.log` on the server. Then I ran `strace sshd`, e.g. strace --follow-forks --output-separately -o /tmp/sshd-9.5p1.out -- /opt/openssh-9.5p1/sbin/sshd -D Elsewhere, login with an ssh client. The connection fails at the same `debug1: SSH2_MSG_KEXINIT sent`. The `auth.log` has messages: 2023-12-02T20:01:24.242176-08:00 host1 sshd[5905]: debug3: fd 5 is not O_NONBLOCK 2023-12-02T20:01:24.242601-08:00 host1 sshd[5905]: debug1: Forked child 5951. 2023-12-02T20:01:24.242819-08:00 host1 sshd[5905]: debug3: send_rexec_state: entering fd = 8 config len 3236 2023-12-02T20:01:24.243968-08:00 host1 sshd[5951]: debug3: oom_adjust_restore 2023-12-02T20:01:24.244254-08:00 host1 sshd[5905]: debug3: ssh_msg_send: type 0 2023-12-02T20:01:24.244408-08:00 host1 sshd[5905]: debug3: send_rexec_state: done 2023-12-02T20:01:24.244559-08:00 host1 sshd[5951]: debug1: Set /proc/self/oom_score_adj to 0 2023-12-02T20:01:24.244726-08:00 host1 sshd[5951]: debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8 2023-12-02T20:01:24.268631-08:00 host1 sshd[5951]: debug1: inetd sockets after dupping: 4, 4 2023-12-02T20:01:24.269001-08:00 host1 sshd[5951]: debug3: process_channel_timeouts: setting 0 timeouts 2023-12-02T20:01:24.269321-08:00 host1 sshd[5951]: debug3: channel_clear_timeouts: clearing 2023-12-02T20:01:24.269677-08:00 host1 sshd[5951]: Connection from 10.0.1.2 port 42270 on 192.168.1.2 port 2234 rdomain "" 2023-12-02T20:01:24.269973-08:00 host1 sshd[5951]: debug1: Local version string SSH-2.0-OpenSSH_9.5 2023-12-02T20:01:24.270263-08:00 host1 sshd[5951]: debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5+deb11u2 2023-12-02T20:01:24.270561-08:00 host1 sshd[5951]: debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5+deb11u2 pat OpenSSH* compat 0x04000000 2023-12-02T20:01:24.270834-08:00 host1 sshd[5951]: debug2: fd 4 setting O_NONBLOCK 2023-12-02T20:01:24.271097-08:00 host1 sshd[5951]: debug3: ssh_sandbox_init: preparing seccomp filter sandbox 2023-12-02T20:01:24.271337-08:00 host1 sshd[5951]: debug2: Network child is on pid 5952 2023-12-02T20:01:24.271573-08:00 host1 sshd[5951]: debug3: preauth child monitor started 2023-12-02T20:01:24.272750-08:00 host1 sshd[5951]: debug3: privsep user:group 107:65534 [preauth] 2023-12-02T20:01:24.273121-08:00 host1 sshd[5951]: debug1: permanently_set_uid: 107/65534 [preauth] 2023-12-02T20:01:24.273369-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child_debugging: installing SIGSYS handler [preauth] 2023-12-02T20:01:24.273680-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] 2023-12-02T20:01:24.273938-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] 2023-12-02T20:01:24.274355-08:00 host1 sshd[5951]: debug1: monitor_read_log: child log fd closed 2023-12-02T20:01:24.274650-08:00 host1 sshd[5951]: debug3: mm_request_receive: entering 2023-12-02T20:01:24.274954-08:00 host1 sshd[5951]: debug1: do_cleanup 2023-12-02T20:01:24.275832-08:00 host1 sshd[5951]: debug1: Killing privsep child 5952 2023-12-02T20:01:24.241307-08:00 host1 sshd[5905]: debug3: fd 5 is not O_NONBLOCK 2023-12-02T20:01:24.242538-08:00 host1 sshd[5905]: debug1: Forked child 5951. 2023-12-02T20:01:24.242765-08:00 host1 sshd[5905]: debug3: send_rexec_state: entering fd = 8 config len 3236 2023-12-02T20:01:24.242956-08:00 host1 sshd[5951]: debug3: oom_adjust_restore 2023-12-02T20:01:24.244193-08:00 host1 sshd[5905]: debug3: ssh_msg_send: type 0 2023-12-02T20:01:24.244372-08:00 host1 sshd[5905]: debug3: send_rexec_state: done 2023-12-02T20:01:24.244523-08:00 host1 sshd[5951]: debug1: Set /proc/self/oom_score_adj to 0 2023-12-02T20:01:24.244690-08:00 host1 sshd[5951]: debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8 2023-12-02T20:01:24.268540-08:00 host1 sshd[5951]: debug1: inetd sockets after dupping: 4, 4 2023-12-02T20:01:24.268939-08:00 host1 sshd[5951]: debug3: process_channel_timeouts: setting 0 timeouts 2023-12-02T20:01:24.269262-08:00 host1 sshd[5951]: debug3: channel_clear_timeouts: clearing 2023-12-02T20:01:24.269549-08:00 host1 sshd[5951]: Connection from 10.0.1.2 port 42270 on 192.168.1.2 port 2234 rdomain "" 2023-12-02T20:01:24.269909-08:00 host1 sshd[5951]: debug1: Local version string SSH-2.0-OpenSSH_9.5 2023-12-02T20:01:24.270201-08:00 host1 sshd[5951]: debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Debian-5+deb11u2 2023-12-02T20:01:24.270504-08:00 host1 sshd[5951]: debug1: compat_banner: match: OpenSSH_8.4p1 Debian-5+deb11u2 pat OpenSSH* compat 0x04000000 2023-12-02T20:01:24.270782-08:00 host1 sshd[5951]: debug2: fd 4 setting O_NONBLOCK 2023-12-02T20:01:24.271048-08:00 host1 sshd[5951]: debug3: ssh_sandbox_init: preparing seccomp filter sandbox 2023-12-02T20:01:24.271290-08:00 host1 sshd[5951]: debug2: Network child is on pid 5952 2023-12-02T20:01:24.271528-08:00 host1 sshd[5951]: debug3: preauth child monitor started 2023-12-02T20:01:24.272669-08:00 host1 sshd[5951]: debug3: privsep user:group 107:65534 [preauth] 2023-12-02T20:01:24.273066-08:00 host1 sshd[5951]: debug1: permanently_set_uid: 107/65534 [preauth] 2023-12-02T20:01:24.273325-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child_debugging: installing SIGSYS handler [preauth] 2023-12-02T20:01:24.273623-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] 2023-12-02T20:01:24.273885-08:00 host1 sshd[5951]: debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] 2023-12-02T20:01:24.274293-08:00 host1 sshd[5951]: debug1: monitor_read_log: child log fd closed 2023-12-02T20:01:24.274593-08:00 host1 sshd[5951]: debug3: mm_request_receive: entering 2023-12-02T20:01:24.274892-08:00 host1 sshd[5951]: debug1: do_cleanup 2023-12-02T20:01:24.275192-08:00 host1 sshd[5951]: debug1: Killing privsep child 5952 The third `sshd` process, created during client login has the error near the end of the strace output: $ cat sshd-9.5p1.out.8285 ... prlimit64(0, RLIMIT_FSIZE, {rlim_cur=0, rlim_max=0}, NULL) = 0 prlimit64(0, RLIMIT_NOFILE, {rlim_cur=1, rlim_max=1}, NULL) = 0 prlimit64(0, RLIMIT_NPROC, {rlim_cur=0, rlim_max=0}, NULL) = 0 getpid() = 8285 write(8, "\0\0\0B\0\0\0\7\0\0\0\0\0\0\0006ssh_sandbox_chil"..., 70) = 70 rt_sigaction(SIGSYS, {sa_handler=0x877aec, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO, sa_restorer=0xf7a40910}, NULL, 8) = 0 rt_sigprocmask(SIG_UNBLOCK, [SYS], NULL, 8) = 0 getpid() = 8285 write(8, "\0\0\0:\0\0\0\7\0\0\0\0\0\0\0.ssh_sandbox_chil"..., 62) 62 prctl(PR_SET_NO_NEW_PRIVS, 1, 0, 0, 0) = 0 getpid() = 8285 write(8, "\0\0\0?\0\0\0\7\0\0\0\0\0\0\0003ssh_sandbox_chil"..., 67) = 67 prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, {len=276, filter=0x911db0}) = 0 getpid() = 0 --- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0xf7ab480c, si_syscall=__NR_getpid, si_arch=AUDIT_ARCH_ARM} --- write(8, "\0\0\0c\0\0\0\1\0\0\0\0\0\0\0Wssh_sandbox_viol"..., 103) = 8 You can see that last `write` message with substring value `ssh_sandbox_viol`. Running `strace sshd` again without abbreviations, e.g. strace -tt --no-abbrev --follow-forks --output-separately -o /tmp/sshd-9.5p1.out -- /opt/openssh-9.5p1/sbin/sshd -D In the erroring child process, that last `prctl` call just before the `ssh_sandbox_viol` message is: 20:44:11.669293 write(8, "\0\0\0?\0\0\0\7\0\0\0\0\0\0\0003ssh_sandbox_chil"..., 67) = 67 20:44:11.669470 prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, {len=276, filter=[BPF_STMT(BPF_LD|BPF_W|BPF_ABS, 0x4), ...}) = 0 20:44:11.670963 getpid() = 0 20:44:11.671139 --- SIGSYS {si_signo=SIGSYS, si_code=SYS_SECCOMP, si_call_addr=0xf779180c, si_syscall=__NR_getpid, si_arch=AUDIT_ARCH_ARM} --- 20:44:11.671218 write(8, "\0\0\0c\0\0\0\1\0\0\0\0\0\0\0Wssh_sandbox_viol"..., 103) = 8 The full `prctl` function call is in attachment `prctl-PR_SET_SECCOMP.txt` Running `strace sshd` with longer strings: strace -tt --no-abbrev -s 128 --follow-forks --output-separately -o /tmp/sshd-9.5p1.out -- /opt/openssh-9.5p1/sbin/sshd -D The full string `write` is: 21:01:41.819827 write(8, "\0\0\0c\0\0\0\1\0\0\0\0\0\0\0Wssh_sandbox_violation: unexpected system call (arch:0x40000028,syscall:20 @ 0xf7ba380c)", 103) = 8 -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-03 05:14 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #3 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3763 --> https://bugzilla.mindrot.org/attachment.cgi?id=3763&action=edit full formatted output of prctl(PR_SET_SECCOMP, ...) -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-03 05:24 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 JM <jtm.moon.forum.user+mindrot at gmail.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Summary|server thread aborts during |server thread aborts during |client login after |client login after |receiving SSH2_MSG_KEXINIT |receiving SSH2_MSG_KEXINIT | |due to | |ssh_sandbox_violation -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 03:14 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 Damien Miller <djm at mindrot.org> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |djm at mindrot.org --- Comment #4 from Damien Miller <djm at mindrot.org> --- This is the details of the sandbox violation:> ssh_sandbox_violation: unexpected system call (arch:0x40000028,syscall:20 @ 0xf7ba380csyscall 20 is getpid:> [djm at djm linux]$ grep 'NR.* 20$' arch/arm64/include/asm/unistd32.h > #define __NR_getpid 20but getpid is allowed by the sshd sandbox policy:> [djm at djm openssh]$ grep -A1 getpid sandbox-seccomp-filter.c > #ifdef __NR_getpid > SC_ALLOW(__NR_getpid), > #endifHowever, this only works in __NR_getpid is defined in a system header than that header is correctly picked up during sshd's compilation. If your system headers are messed up then sshd won't pick up the correct syscall number and sandbox violations will occur. You could try attaching ./configure output and config.h (please don't paste them inline), which might help, but I suspect that the root cause is going that your system headers are messed up in some way. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 04:19 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #5 from Darren Tucker <dtucker at dtucker.net> --- (In reply to Damien Miller from comment #4)> This is the details of the sandbox violation: > > > ssh_sandbox_violation: unexpected system call (arch:0x40000028,syscall:20 @ 0xf7ba380c > > syscall 20 is getpid: > > > [djm at djm linux]$ grep 'NR.* 20$' arch/arm64/include/asm/unistd32.h > > #define __NR_getpid 20That's not what it is on my rpi4. I think that's for 32bit ARM. $ uname -a Linux hostname 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux $ grep -r -E '__NR.*20$' /usr/include /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 $ grep -E NR.*getpid /usr/include/asm-generic/unistd.h #define __NR_getpid 172 which a test program confirms: $ cat test.c #include <stdio.h> #include <syscall.h> int main(void) { printf("__NR_epoll_create1 %d\n", __NR_epoll_create1); printf("__NR_getpid %d\n", __NR_getpid); } $ cc test.c && ./a.out __NR_epoll_create1 20 __NR_getpid 172 Testing on a 32bit arm, that is indeed 20: $ uname -a Linux hostname 5.16.10-bone14 #1bullseye PREEMPT Tue Feb 22 00:07:39 UTC 2022 armv7l GNU/Linux $ cc test.c && ./a.out __NR_epoll_create1 357 __NR_getpid 20 So perhaps the problem here is that either it's picking up 32bit vs 64bit headers, or that the binary is some kind of 32bit compatibility mode but the sandbox is expecting the 64bit syscalls. -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 04:23 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #6 from Darren Tucker <dtucker at dtucker.net> --- JM: what compiler are you using? Another thing that might be interesting is to compare config.h and the output of configure from 9.1p1 with those of 9.2p1 and see if anything unexpectedly changed. -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 04:30 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #7 from Darren Tucker <dtucker at dtucker.net> --- Created attachment 3764 --> https://bugzilla.mindrot.org/attachment.cgi?id=3764&action=edit configure output from 9.2p1 working on rpi4 -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 04:31 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #8 from Darren Tucker <dtucker at dtucker.net> --- Created attachment 3765 --> https://bugzilla.mindrot.org/attachment.cgi?id=3765&action=edit config.h from 9.2p1 working on rpi4 here's the configure output and config.h from my working system for comparison. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-04 22:45 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #9 from Damien Miller <djm at mindrot.org> --- hmm, it looks like I might have been wrong with the syscall number:> [djm at djm linux]$ grep ' 20$' include/uapi/asm-generic/unistd.h > #define __NR_epoll_create1 20It's likely that something is trying to use the epoll(3) API. OpenSSH itself doesn't use epoll, so it's likely to be something in libc, libcrypto or another library. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-05 00:39 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #10 from Darren Tucker <dtucker at dtucker.net> --- (In reply to Damien Miller from comment #9)> It's likely that something is trying to use the epoll(3) API. > OpenSSH itself doesn't use epoll, so it's likely to be something in > libc, libcrypto or another library.That's possible. I checked /etc/os-release on my device, and it's stock Debian not Raspbian. My guess would be something in libcrypto, in which case configuring --without-openssl and retesting would be a good indicator). -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-05 23:41 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #11 from Darren Tucker <dtucker at dtucker.net> --- I'd like to reproduce this locally. Could you please attach /etc/os-release and the output of "dpkg -l" from the affected device? -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:10 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #12 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- tl;dr compiles and runs okay on a Raspberry Pi3 and NanoPi NEO3 running similar OS For comparison, I've included three similar, not the same, platforms: - Raspberry Pi 4 (RPi4) (aarch64) (Raspbian) on which this bug report originated - Raspberry Pi 3 (RPi3) (armv7l) (Raspbian) on which openssh 9.5p1 runs fine (client can login, no thread aborts) - NanoPi NEO3 Rockchip RK3288 (aarch64) (Debian) on which openssh 9.5p1 runs fine (client can login, no thread aborts) I attached three ".info" files with various dumps of info about each system. To be clearer about the OS, I'm using the DietPI-managed distribution of Raspbian (for Raspberry Pis) or Debian (for NanoPi) (https://dietpi.com/).> JM: what compiler are you using?On RPi4 (has aborts) $ cc --version cc (Raspbian 10.2.1-6+rpi1) 10.2.1 20210110 On RPi3 (runs okay) $ cc --version cc (Raspbian 10.2.1-6+rpi1) 10.2.1 20210110 On NanoPi NEO3 (runs okay) $ cc --version cc (Debian 10.2.1-6) 10.2.1 20210110 On RPi4 (has aborts) $ grep -r -E '__NR.*20$' /usr/include /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:#define __NR_sched_getscheduler 120 /usr/include/asm-generic/unistd.h:#define __NR_clone 220 /usr/include/asm-generic/unistd.h:#define __NR_semtimedop_time64 420 $ grep -E NR.*getpid /usr/include/asm-generic/unistd.h #define __NR_getpid 172 __SYSCALL(__NR_getpid, sys_getpid) On RPi3 (runs okay) $ grep -r -E '__NR.*20$' /usr/include /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:#define __NR_sched_getscheduler 120 /usr/include/asm-generic/unistd.h:#define __NR_clone 220 /usr/include/asm-generic/unistd.h:#define __NR_semtimedop_time64 420 $ grep -E NR.*getpid /usr/include/asm-generic/unistd.h #define __NR_getpid 172 __SYSCALL(__NR_getpid, sys_getpid) On NanoPi NEO3 (runs okay) $ grep -r -E '__NR.*20$' /usr/include /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:#define __NR_sched_getscheduler 120 /usr/include/asm-generic/unistd.h:#define __NR_clone 220 /usr/include/asm-generic/unistd.h:#define __NR_semtimedop_time64 420 $ grep -E NR.*getpid /usr/include/asm-generic/unistd.h #define __NR_getpid 172 __SYSCALL(__NR_getpid, sys_getpid)> which a test program confirms:Results from RPi4 (has aborts), using the same `test.c` source code show above. $ cc test.c $ ./a.out __NR_epoll_create1 357 __NR_getpid 20 On RPi3 (runs okay) same results: $ cc test.c $ ./a.out __NR_epoll_create1 357 __NR_getpid 20 On NanoPi NEO3 (runs okay) differs: $ cc test.c $ ./a.out __NR_epoll_create1 20 __NR_getpid 172> Another thing that might be interesting is to compare config.h > and the output of configure from 9.1p1 with those of 9.2p1 and > see if anything unexpectedly changed.Compared on the RPi4 (has aborts) of 9.1p1 and 9.2p1: $ diff -d -B -W 80 -y --suppress-common-lines -- config-9.1p1.h config-9.2p1.h #define DISABLE_WTMPX 1 | /* #undef DISABLE_WTMPX */ #define HAVE_DECL_MEMMEM 0 | #define HAVE_DECL_MEMMEM 1 /* #undef HAVE_SIGHANDLER_T */ | #define HAVE_SIGHANDLER_T 1 > /* sockaddr_in has sin_len */ > /* #undef SOCK_HAS_LEN */ #define USER_PATH "/usr/bin:/bin:/usr | #define USER_PATH "/usr/bin:/bin:/usr Attached both config.h files from RPi4; `config-9.1p1.h`, `config-9.2p1.h`.> Could you please attach /etc/os-release and the output of "dpkg -l" > from the affected device?Attached in `RPi4-dpkg-l.txt` and `RPi4.info`.> My guess would be something in libcrypto, in which case > configuring --without-openssl and retesting would be a good indicator).Tried `--without-openssl` on RPi4 using 9.5p1. Same error occurs. $ make clean $ ./configure --prefix=/opt/openssh-9.5p1-noopenssl --without-openssl $ make $ make install --- In case anyone is curious, general installation steps are taken from my gist https://gist.github.com/jtmoon79/745e6df63dd14b9f2d17a662179e953a --James T Moon (https://github.com/jtmoon79/) -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:11 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #13 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3766 --> https://bugzilla.mindrot.org/attachment.cgi?id=3766&action=edit config-9.1p1.h -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:12 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #14 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3767 --> https://bugzilla.mindrot.org/attachment.cgi?id=3767&action=edit config-9.2p1.h -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:12 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #15 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3768 --> https://bugzilla.mindrot.org/attachment.cgi?id=3768&action=edit RPi4-dpkg-l.txt -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:12 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #16 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3769 --> https://bugzilla.mindrot.org/attachment.cgi?id=3769&action=edit RPi3-dpkg-l.txt -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:13 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #17 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3770 --> https://bugzilla.mindrot.org/attachment.cgi?id=3770&action=edit NanoPi_NEO3.info -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:13 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #18 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3771 --> https://bugzilla.mindrot.org/attachment.cgi?id=3771&action=edit RPi3.info -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:13 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #19 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3772 --> https://bugzilla.mindrot.org/attachment.cgi?id=3772&action=edit RPi4.info -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:19 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #20 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3773 --> https://bugzilla.mindrot.org/attachment.cgi?id=3773&action=edit NanoPi-dpkg-l.txt -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-10 23:29 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #21 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- A little more info about `__NR_getpid` and `__NR_epoll_create1` (not sure if this is relevant but in case you were curious) On RPi4 (has aborts) $ grep -r -Ee '__NR_getpid|__NR_epoll_create1' -- /usr/include/ /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_epoll_create1, sys_epoll_create1) /usr/include/asm-generic/unistd.h:#define __NR_getpid 172 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_getpid, sys_getpid) /usr/include/arm-linux-gnueabihf/bits/syscall.h:#ifdef __NR_epoll_create1 /usr/include/arm-linux-gnueabihf/bits/syscall.h:# define SYS_epoll_create1 __NR_epoll_create1 /usr/include/arm-linux-gnueabihf/bits/syscall.h:#ifdef __NR_getpid /usr/include/arm-linux-gnueabihf/bits/syscall.h:# define SYS_getpid __NR_getpid /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h:#define __NR_getpid (__NR_SYSCALL_BASE + 20) /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h:#define __NR_epoll_create1 (__NR_SYSCALL_BASE + 357) /usr/include/arm-linux-gnueabihf/asm/unistd-oabi.h:#define __NR_getpid (__NR_SYSCALL_BASE + 20) /usr/include/arm-linux-gnueabihf/asm/unistd-oabi.h:#define __NR_epoll_create1 (__NR_SYSCALL_BASE + 357) On RPi3 (runs okay) $ grep -r -Ee '__NR_getpid|__NR_epoll_create1' -- /usr/include/ /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_epoll_create1, sys_epoll_create1) /usr/include/asm-generic/unistd.h:#define __NR_getpid 172 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_getpid, sys_getpid) /usr/include/arm-linux-gnueabihf/bits/syscall.h:#ifdef __NR_epoll_create1 /usr/include/arm-linux-gnueabihf/bits/syscall.h:# define SYS_epoll_create1 __NR_epoll_create1 /usr/include/arm-linux-gnueabihf/bits/syscall.h:#ifdef __NR_getpid /usr/include/arm-linux-gnueabihf/bits/syscall.h:# define SYS_getpid __NR_getpid /usr/include/arm-linux-gnueabihf/asm/unistd-oabi.h:#define __NR_getpid (__NR_SYSCALL_BASE + 20) /usr/include/arm-linux-gnueabihf/asm/unistd-oabi.h:#define __NR_epoll_create1 (__NR_SYSCALL_BASE + 357) /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h:#define __NR_getpid (__NR_SYSCALL_BASE + 20) /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h:#define __NR_epoll_create1 (__NR_SYSCALL_BASE + 357) On NanoPi NEO3 (runs okay) $ grep -r -Ee '__NR_getpid|__NR_epoll_create1' -- /usr/include/ /usr/include/asm-generic/unistd.h:#define __NR_epoll_create1 20 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_epoll_create1, sys_epoll_create1) /usr/include/asm-generic/unistd.h:#define __NR_getpid 172 /usr/include/asm-generic/unistd.h:__SYSCALL(__NR_getpid, sys_getpid) /usr/include/aarch64-linux-gnu/bits/syscall.h:#ifdef __NR_epoll_create1 /usr/include/aarch64-linux-gnu/bits/syscall.h:# define SYS_epoll_create1 __NR_epoll_create1 /usr/include/aarch64-linux-gnu/bits/syscall.h:#ifdef __NR_getpid /usr/include/aarch64-linux-gnu/bits/syscall.h:# define SYS_getpid __NR_getpid -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-11 05:25 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #22 from Damien Miller <djm at mindrot.org> --- Created attachment 3774 --> https://bugzilla.mindrot.org/attachment.cgi?id=3774&action=edit syscall dumper Could you try building and running this program. E.g. $ cc -o syscall syscall.c $ ./syscall $ strace -n ./syscall This will tell you what the compiler thinks __NR_getpid is vs what the actual syscall is. -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-11 05:26 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #23 from Damien Miller <djm at mindrot.org> --- i.e. run it on a platform that works and the one that doesn't -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-11 05:45 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #24 from JM <jtm.moon.forum.user+mindrot at gmail.com> ---> Could you try building and running this program. E.g. > > $ cc -o syscall syscall.c > $ ./syscall > $ strace -n ./syscallRaspberry Pi 4 (RPi4), aarch64, Raspbian-Debian 11, (openssh 9.5p1 client thread aborts): $ (set -eux; wget -q "https://bugzilla.mindrot.org/attachment.cgi?id=3774" -O syscall.c ; cc -o syscall syscall.c ; ./syscall ; strace -n ./syscall ) + wget -q 'https://bugzilla.mindrot.org/attachment.cgi?id=3774' -O syscall.c + cc -o syscall syscall.c + ./syscall __NR_getpid = 20 + strace -n ./syscall [ 11] execve("./syscall", ["./syscall"], 0xffb2b5b4 /* 31 vars */) = 0 [ 45] brk(NULL) = 0x1ae5000 [ 122] uname({sysname="Linux", nodename="pifuboo", ...}) = 0 [ 33] access("/etc/ld.so.preload", R_OK) = 0 [ 322] openat(AT_FDCWD, "/etc/ld.so.preload", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0 [ 192] mmap2(NULL, 54, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) 0xf7d7f000 [ 6] close(3) = 0 [ 85] readlink("/proc/self/exe", "/tmp/syscall", 4096) = 12 [ 322] openat(AT_FDCWD, "/usr/lib/arm-linux-gnueabihf/libarmmem-v8l.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 3] read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\254\3\0\0004\0\0\0"..., 512) = 512 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=17708, ...}) = 0 [ 192] mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xf7d7d000 [ 192] mmap2(NULL, 81964, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf7d3a000 [ 125] mprotect(0xf7d3e000, 61440, PROT_NONE) = 0 [ 192] mmap2(0xf7d4d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0xf7d4d000 [ 6] close(3) = 0 [ 91] munmap(0xf7d7f000, 54) = 0 [ 322] openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=63463, ...}) = 0 [ 192] mmap2(NULL, 63463, PROT_READ, MAP_PRIVATE, 3, 0) 0xf7d2a000 [ 6] close(3) = 0 [ 322] openat(AT_FDCWD, "/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 3] read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0y\1\0004\0\0\0"..., 512) = 512 [ 197] fstat64(3, {st_mode=S_IFREG|0755, st_size=1315688, ...}) = 0 [ 192] mmap2(NULL, 1385020, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xf7bd7000 [ 125] mprotect(0xf7d15000, 61440, PROT_NONE) = 0 [ 192] mmap2(0xf7d24000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13d000) = 0xf7d24000 [ 192] mmap2(0xf7d27000, 8764, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xf7d27000 [ 6] close(3) = 0 [983045] set_tls(0xf7d7df80) = 0 [ 125] mprotect(0xf7d24000, 8192, PROT_READ) = 0 [ 125] mprotect(0xf7d4d000, 4096, PROT_READ) = 0 [ 125] mprotect(0x20000, 4096, PROT_READ) = 0 [ 125] mprotect(0xf7d81000, 4096, PROT_READ) = 0 [ 91] munmap(0xf7d2a000, 63463) = 0 [ 197] fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0 [ 45] brk(NULL) = 0x1ae5000 [ 45] brk(0x1b06000) = 0x1b06000 [ 4] write(1, "__NR_getpid = 20\n", 17__NR_getpid = 20 ) = 17 [ 20] getpid() = 21435 [ 248] exit_group(0) = ? [ 248] +++ exited with 0 +++ Raspberry Pi 3 (RPi3), armv7l, Raspbian Debian 11, (openssh 9.5p1 runs okay): $ (set -eux; wget -q "https://bugzilla.mindrot.org/attachment.cgi?id=3774" -O syscall.c ; cc -o syscall syscall.c ; ./syscall ; strace -n ./syscall ) + wget -q 'https://bugzilla.mindrot.org/attachment.cgi?id=3774' -O syscall.c + cc -o syscall syscall.c + ./syscall __NR_getpid = 20 + strace -n ./syscall [ 11] execve("./syscall", ["./syscall"], 0x7eab2584 /* 30 vars */) = 0 [ 45] brk(NULL) = 0x1435000 [ 192] mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x76fa5000 [ 33] access("/etc/ld.so.preload", R_OK) = 0 [ 322] openat(AT_FDCWD, "/etc/ld.so.preload", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=54, ...}) = 0 [ 192] mmap2(NULL, 54, PROT_READ|PROT_WRITE, MAP_PRIVATE, 3, 0) 0x76fa4000 [ 6] close(3) = 0 [ 85] readlink("/proc/self/exe", "/tmp/syscall", 4096) = 12 [ 322] openat(AT_FDCWD, "/usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 3] read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\254\3\0\0004\0\0\0"..., 512) = 512 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=17708, ...}) = 0 [ 192] mmap2(NULL, 81964, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x76f60000 [ 125] mprotect(0x76f64000, 61440, PROT_NONE) = 0 [ 192] mmap2(0x76f73000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x76f73000 [ 6] close(3) = 0 [ 91] munmap(0x76fa4000, 54) = 0 [ 322] openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 197] fstat64(3, {st_mode=S_IFREG|0644, st_size=55352, ...}) = 0 [ 192] mmap2(NULL, 55352, PROT_READ, MAP_PRIVATE, 3, 0) 0x76f97000 [ 6] close(3) = 0 [ 322] openat(AT_FDCWD, "/lib/arm-linux-gnueabihf/libc.so.6", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3 [ 3] read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0(\0\1\0\0\0\0y\1\0004\0\0\0"..., 512) = 512 [ 197] fstat64(3, {st_mode=S_IFREG|0755, st_size=1315688, ...}) = 0 [ 192] mmap2(NULL, 1385020, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x76e0d000 [ 125] mprotect(0x76f4b000, 61440, PROT_NONE) = 0 [ 192] mmap2(0x76f5a000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13d000) = 0x76f5a000 [ 192] mmap2(0x76f5d000, 8764, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x76f5d000 [ 6] close(3) = 0 [983045] set_tls(0x76fa6280) = 0 [ 125] mprotect(0x76f5a000, 8192, PROT_READ) = 0 [ 125] mprotect(0x76f73000, 4096, PROT_READ) = 0 [ 125] mprotect(0x20000, 4096, PROT_READ) = 0 [ 125] mprotect(0x76fa7000, 4096, PROT_READ) = 0 [ 91] munmap(0x76f97000, 55352) = 0 [ 197] fstat64(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0 [ 45] brk(NULL) = 0x1435000 [ 45] brk(0x1456000) = 0x1456000 [ 4] write(1, "__NR_getpid = 20\n", 17__NR_getpid = 20 ) = 17 [ 20] getpid() = 19383 [ 248] exit_group(0) = ? [ 248] +++ exited with 0 +++ NanoPi NEO3 Rockchip RK3288, aarch64, Debian, (openssh 9.5p1 runs okay): $ (set -eux; wget -q "https://bugzilla.mindrot.org/attachment.cgi?id=3774" -O syscall.c ; cc -o syscall syscall.c ; ./syscall ; strace -n ./syscall ) + wget -q 'https://bugzilla.mindrot.org/attachment.cgi?id=3774' -O syscall.c + cc -o syscall syscall.c + ./syscall __NR_getpid = 172 + strace -n ./syscall [ 221] execve("./syscall", ["./syscall"], 0xffffed56ea98 /* 27 vars */) = 0 [ 214] brk(NULL) = 0xaaaae9498000 [ 48] faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) [ 56] openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 [ 80] fstat(3, {st_mode=S_IFREG|0644, st_size=24779, ...}) = 0 [ 222] mmap(NULL, 24779, PROT_READ, MAP_PRIVATE, 3, 0) 0xffffa86b7000 [ 57] close(3) = 0 [ 56] openat(AT_FDCWD, "/lib/aarch64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 [ 63] read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0`\17\2\0\0\0\0\0"..., 832) = 832 [ 80] fstat(3, {st_mode=S_IFREG|0755, st_size=1451024, ...}) = 0 [ 222] mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xffffa86b5000 [ 222] mmap(NULL, 1523656, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xffffa851c000 [ 226] mprotect(0xffffa8678000, 61440, PROT_NONE) = 0 [ 222] mmap(0xffffa8687000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15b000) = 0xffffa8687000 [ 222] mmap(0xffffa868d000, 12232, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xffffa868d000 [ 57] close(3) = 0 [ 226] mprotect(0xffffa8687000, 16384, PROT_READ) = 0 [ 226] mprotect(0xaaaaacbf0000, 4096, PROT_READ) = 0 [ 226] mprotect(0xffffa86c1000, 4096, PROT_READ) = 0 [ 215] munmap(0xffffa86b7000, 24779) = 0 [ 80] fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}) = 0 [ 214] brk(NULL) = 0xaaaae9498000 [ 214] brk(0xaaaae94b9000) = 0xaaaae94b9000 [ 64] write(1, "__NR_getpid = 172\n", 18__NR_getpid = 172 ) = 18 [ 172] getpid() = 7407 [ 94] exit_group(0) = ? [ 94] +++ exited with 0 +++ -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-11 10:09 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #25 from Damien Miller <djm at mindrot.org> --- ok, so now I have no idea what is going wrong. Maybe there is something in OpenSSH's compile flags that is messing this up. Could you add try adding a similar printf+getpid+exit sequence to (say) the start of ssh-keygen.c, rerunning make and repeating the test using the resultant binary? You probably only need to do this on the rpi4 -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-18 07:00 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #26 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- tl;dr a seccomp sandbox violation `20` occurs from a `read` (still). This is just a more detailed retelling of what was previously discussed. Scroll to end for thoughts... ### problem specifics Failed `read` in parent process read(fd, s + pos, n - pos); which is read(5, '\014\221b', 4); returns `0`. Failed `read` will cause the following audit event (journalctl -f -x) Dec 17 15:59:35 host1 kernel: audit: type=1326 audit(1702857575.824:3180): auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=1920 comm="sshd" exe="/root/Projects/openssh-9.2p1-WIP/sshd" sig=31 arch=40000028 syscall=20 compat=1 ip=0xf7afd80c code=0x0 And the same when compiled with `CFLAGS="-DDSANDBOX_SECCOMP_FILTER_DEBUG"` Dec 17 22:37:50 pifuboo audit[10678]: SECCOMP auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=10678 comm="sshd" exe="/root/Projects/openssh-9.2p1-WIP/sshd" sig=31 arch=40000028 syscall=20 compat=1 ip=0xf77de80c code=0x0 Dec 17 22:37:50 pifuboo audit[10678]: ANOM_ABEND auid=4294967295 uid=107 gid=65534 ses=4294967295 pid=10678 comm="sshd" exe="/root/Projects/openssh-9.2p1-WIP/sshd" sig=31 res=1 The failed linux system call `20` is `epoll_create1` according to `ausyscall` $ ausyscall 20 epoll_create1 So the `read` at some point calls syscall `20`. See section "Summary Thoughts" about this. Here is the failed `read` call https://github.com/openssh/openssh-portable/blob/V_9_2_P1/atomicio.c#L66 It is always the `read` call with values `fd=5`, `n=4`. `read` returns `0`. `errno` is not changed after `read` returns. The stack just before the failed `read` call is: #1 0x004701f8 in atomicio6 (f=f at entry=0xf7c65478 <read>, fd=fd at entry=5, _s=0xfffeead8, _s at entry=0xfffeead0, n=n at entry=4, cb=cb at entry=0x0, cb_arg=cb_arg at entry=0x0) at atomicio.c:67 #2 0x00470284 in atomicio (f=f at entry=0xf7c65478 <read>, fd=fd at entry=5, _s=_s at entry=0xfffeead0, n=n at entry=4) at atomicio.c:101 #3 0x00434520 in mm_request_receive (sock=5, m=m at entry=0x4f2b88) at monitor_wrap.c:149 #4 0x00431178 in monitor_read (ssh=ssh at entry=0x4f3388, pmonitor=pmonitor at entry=0x4f2498, ent=0x4e0114 <mon_dispatch_proto20>, pent=pent at entry=0xfffeeb78) at monitor.c:501 #5 0x00433b5c in monitor_child_preauth (ssh=ssh at entry=0x4f3388, pmonitor=0x4f2498) at monitor.c:301 #6 0x0040a388 in privsep_preauth (ssh=0x4f3388) at sshd.c:502 #7 main (ac=<optimized out>, av=0x4e31a0) at sshd.c:2240 (line numbers in `atomicio.c` slightly off due to insertion of `raise(SIGINT);`) The debug log from the start of client connection is: debug3: fd 5 is not O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 3296 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 process 32277 is executing new program: /root/Projects/openssh-9.2p1/sshd [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1". debug3: recv_rexec_state: entering fd = 5 debug3: ssh_msg_recv entering debug3: recv_rexec_state: done debug2: parse_server_config_depth: config rexec len 3296 debug3: rexec:14 setting Port 55222 debug3: rexec:22 setting HostKey /root/Projects/openssh-9.2p1/ssh_host_ecdsa_key debug3: rexec:23 setting HostKey /root/Projects/openssh-9.2p1/ssh_host_ed25519_key debug3: rexec:24 setting HostKey /root/Projects/openssh-9.2p1/ssh_host_rsa_key debug3: rexec:45 setting AuthorizedKeysFile .ssh/authorized_keys debug3: rexec:113 setting Subsystem sftp /usr/libexec/sftp-server debug1: sshd version OpenSSH_9.2, OpenSSL 1.1.1w 11 Sep 2023 debug1: private host key #0: ecdsa-sha2-nistp256 SHA256:7OUDaY7vmsaJPDkqGWPmdiw5kjY4bVSwCd94nJqT7/o debug1: private host key #1: ssh-ed25519 SHA256:CuPO+bnbHMCkaNEybTHeYSjdNpiNdAlntO9gh0V9lxs debug1: private host key #2: ssh-rsa SHA256:ZYZLLhbWdOMFKDGw3pcn954Wz6RhwtDoI5WjJsZpXhk debug1: inetd sockets after dupping: 3, 3 debug3: process_channel_timeouts: setting 0 timeouts debug3: channel_clear_timeouts: clearing Connection from 192.168.124.214 port 57930 on 192.168.124.214 port 55222 rdomain "" debug1: Local version string SSH-2.0-OpenSSH_9.2 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.4p1 Raspbian-5+deb11u2 debug1: compat_banner: match: OpenSSH_8.4p1 Raspbian-5+deb11u2 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug3: ssh_sandbox_init: preparing seccomp filter sandbox [Detaching after fork from child process 32308] debug2: Network child is on pid 32308 debug3: preauth child monitor started debug3: privsep user:group 107:65534 [preauth] debug1: permanently_set_uid: 107/65534 [preauth] debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive: entering Dumping `/proc/$parentpid/status` just before the `read` failure shows: Seccomp: 0 Seccomp_filters: 0 Dumping `/proc/$childpid/status` just before the `read` failure shows: Seccomp: 3 Seccomp_filters: 1 File descriptor 5 of the parent process is a STREAM (according to `lsof`) $ lsof -p 11715 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME sshd 11715 root cwd DIR 179,2 4096 2 / sshd 11715 root rtd DIR 179,2 4096 2 / sshd 11715 root txt REG 179,2 1318404 72912 /root/Projects/openssh-9.2p1-WIP/sshd sshd 11715 root mem REG 179,2 42628 2737 /lib/arm-linux-gnueabihf/libnss_files-2.31.so sshd 11715 root mem REG 179,2 116324 4913 /lib/arm-linux-gnueabihf/libgcc_s.so.1 sshd 11715 root mem REG 179,2 137364 2748 /lib/arm-linux-gnueabihf/libpthread-2.31.so sshd 11715 root mem REG 179,2 13864 2668 /lib/arm-linux-gnueabihf/libdl-2.31.so sshd 11715 root mem REG 179,2 1315688 2667 /lib/arm-linux-gnueabihf/libc-2.31.so sshd 11715 root mem REG 179,2 95880 11965 /lib/arm-linux-gnueabihf/libz.so.1.2.11 sshd 11715 root mem REG 179,2 2150824 11138 /usr/lib/arm-linux-gnueabihf/libcrypto.so.1.1 sshd 11715 root mem REG 179,2 9796 2753 /lib/arm-linux-gnueabihf/libutil-2.31.so sshd 11715 root mem REG 179,2 210340 1499 /lib/arm-linux-gnueabihf/libcrypt.so.1.1.0 sshd 11715 root mem REG 179,2 17708 14672 /usr/lib/arm-linux-gnueabihf/libarmmem-v7l.so sshd 11715 root mem REG 179,2 146888 2655 /lib/arm-linux-gnueabihf/ld-2.31.so sshd 11715 root 0u CHR 1,3 0t0 5 /dev/null sshd 11715 root 1u CHR 1,3 0t0 5 /dev/null sshd 11715 root 2u CHR 136,4 0t0 7 /dev/pts/4 sshd 11715 root 3u IPv4 8165255 0t0 TCP localhost:55522->localhost:48024 (ESTABLISHED) sshd 11715 root 5u unix 0x00000000c618f590 0t0 8165266 type=STREAM ### Other miscellaneous observations: * the child process quickly becomes "defunct" Oddly, I can see that a child process is created by debug-printing PIDs at certain points. e.g. a debug log message prints debug2: Network child is on pid 11719 But later, just before the failed `read`, that child process is in a "defunct" state. e.g. command `ps -ef` shows $ ps -ef ... sshd: [accepted] [sshd] <defunct> I suspect the child process is immediately dying and the later parent process `read` then fails. * Years ago on this same system, I locally built 8.4p1, 8.6p1, 9.0p1, that have run just fine. 8.4p1 was built Feb 2021 8.6p1 was built Jun 2022 9.0p1 was built Jun 2022 Yet, I downloaded those same old versions today and they failed. Each hit the same child process abort. * I verified the address of `f` is function `read` (https://github.com/openssh/openssh-portable/blob/V_9_2_P1/atomicio.c#L66) with code snippet: if (f == read) { debug3("read(%d, '%s', %u); (errno=%u)", fd, s + pos, n - pos, errno); } else { debug3("(f=%p) (%d, '%s', %u); (errno=%u)", f, fd, s + pos, n - pos, errno); }> Could you add try adding a similar printf+getpid+exit* verified within the `sshd` process, `__NR_epoll_create1 = 357` and `__NR_getpid = 20` via `debug3` prints, e.g. code debug3("__NR_getpid = %d", __NR_getpid); debug3("__NR_epoll_create1 = %d", __NR_epoll_create1); int _pid = getpid(); debug3("getpid() = %d", _pid); * built with `./configure --with-sandbox=no` and it runs okay (no child process aborts) * other sandboxes failed to compile due to missing headers or kernel capabilities (and I didn't feel like chasing these down) * systrace * rlimit * capsicum * Various `fcntl` `GET` checks of file descriptor 5. `errno` was set to `0` before each call to `fcntl`. fcntl(5, F_GETFD) returned 1 (0x00000001) (errno=0) fcntl(5, F_GETOWN_EX) returned 0 (0x00000000) (errno=0) owner.type=0, owner.pid=0 fcntl(5, F_GETOWN) returned 0 (0x00000000) (errno=0) fcntl(5, F_GETPIPE_SZ) returned -1 (0xffffffff) (errno=9) fcntl(5, F_GET_SEALS) returned -1 (0xffffffff) (errno=22) fcntl(5, F_GETLEASE ) returned 2 (0x00000002) (errno=0) * for posterity, if anyone else can repro this, then manually add this code in `atomicio.c` function `atomicio6` to cause a GDB break: if (fd == 5 && n == 4 && pos == 0 && errno == 32) { raise(SIGINT); } Those are the happenstance values before the failed `read` call. Add the prior snippet just before code: res = (f) (fd, s + pos, n - pos); gdb command: $ gdb --args "$(realpath .)/sshd" -ddddd -f sshd_config ### Summary Thoughts> So perhaps the problem here is that either it's picking up 32bit vs 64bit headers, or that the binary is some kind of 32bit compatibility mode but the sandbox is expecting the 64bit syscalls.Looking at the above `grep -r -Ee '__NR_getpid|__NR_epoll_create1'`, maybe the resident compiled libc used header `/usr/include/asm-generic/unistd.h` where `#define __NR_epoll_create1 20`, and openssh-server builds is uses header `/usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h` where `#define __NR_getpid (__NR_SYSCALL_BASE + 20)` (or vice-versa). Looking at the files $ l /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h -rw-r--r-- 1 root root 19938 Apr 5 2023 /usr/include/arm-linux-gnueabihf/asm/unistd-eabi.h $ l /usr/include/asm-generic/unistd.h -rw-r--r-- 1 root root 31480 Apr 5 2023 /usr/include/asm-generic/unistd.h Looking at the compiled `libc` $ find /usr -name 'libc.so' /usr/lib/arm-linux-gnueabihf/libc.so $ ls -l /usr/lib/arm-linux-gnueabihf/libc.so -rw-r--r-- 1 root root 289 Oct 3 12:55 /usr/lib/arm-linux-gnueabihf/libc.so So maybe my include or library pathing is was reconfigured (messed up) some time in October 2023 (oh man, how screwed am I?) ### in the meantime ... I have a workaround using `--with-sandbox=no`. If you'd like me to try something else then please let me know. Otherwise, I've spent a fair amount of time in this rabbit hole and need to get back to other things (i.e. other rabbit holes ??). -James Moon (https://github.com/jtmoon79/) (https://twitter.com/jtmoon1979/) -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-19 02:31 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #27 from Damien Miller <djm at mindrot.org> --- Could you please provide this information so we can try to replicate it ourselves: (In reply to Darren Tucker from comment #11)> I'd like to reproduce this locally. Could you please attach > /etc/os-release and the output of "dpkg -l" from the affected device?Also my request from comment #25:> Could you add try adding a similar printf+getpid+exit sequence to > (say) the start of ssh-keygen.c, rerunning make and repeating the > test using the resultant binary? You probably only need to do this > on the rpi4Also, if you can catch the sandbox-violation in gdb, getting a disassembly of instructions around the violation would be instructive. -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-21 16:52 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #28 from JM <jtm.moon.forum.user+mindrot at gmail.com> ---> I'd like to reproduce this locally. Could you please attach /etc/os-release and the output of "dpkg -l" from the affected device?Attached in `RPi4-dpkg-l.txt` and `RPi4.info`.> Also, if you can catch the sandbox-violation in gdb, getting a disassembly of instructions around the violation would be instructive.I'll give it a shot. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-31 02:57 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #29 from JM <jtm.moon.forum.user+mindrot at gmail.com> ---> Also, if you can catch the sandbox-violation in gdb, getting a disassembly of > instructions around the violation would be instructive.tl;dr reviewing disassembly of a `getpid` call that violates the sandbox shows value `20` is the SYSCALL value used. That SYSCALL value is also in a `SC_ALLOW` BPF filter. ### DEBUG SESSION ### #### Debug Steps #### 1. modify `Makefile` to build with debugging symbols: CFLAGS=-DDSANDBOX_SECCOMP_FILTER_DEBUG -ggdb -O0 ... LDFLAGS=-ggdb -O0 ... 2. In `sandbox-seccomp-filter.c` a. before the call to `prctl(PR_SET_SECCOMP, ...)`, I added a `sleep(20)` and print `getpid()` https://github.com/openssh/openssh-portable/blob/V_9_2_P1/sandbox-seccomp-filter.c#L438 This prints the child process PID, and allows time to manually attach the `gdb` to the new child process. b. Then added call to `raise(SIGINT)` so gdb will break just before `prctl(PR_SET_SECCOMP, ...)`. c. After the call to `prctl(PR_SET_SECCOMP, ...)` I added a call to `getpid()` https://github.com/openssh/openssh-portable/blob/V_9_2_P1/sandbox-seccomp-filter.c#L445 which will predictably invoke the sandbox violation. 3. run the `sshd` program a. modify `sshd_config` to use port 55522 b. build and run (set -eux; make clean; make sshd; "$(realpath .)/sshd" -ddd -f ./sshd_config) 6. run the client ssh to invoke the error ssh -v root at localhost -p 55522 7. when the `sshd` process sleeps for 20, attach `gdb` gdb "--se=$(realpath ./sshd)" --pid=12345 8. in `gdb` a. `continue` it will break at `getpid` just after the `prctl` call b. `stepi` into the call to `getpid` See code in file `bpf-filters-dumped.txt` #### Debug Result #### Here is that run of `gdb` controlling the child process. The child process terminates with `SIGSYS, Bad system call` after call to `getpid` when `getpid` is called *after* the sandbox filters are set by call `prctl(PR_SET_SECCOMP, ...)` https://github.com/openssh/openssh-portable/blob/V_9_2_P1/sandbox-seccomp-filter.c#L439 $ gdb "--se=$(realpath ./sshd)" --pid=16588 ... (gdb) step Single stepping until exit from function raise, which has no line number information. ssh_sandbox_child (box=0x1506010) at sandbox-seccomp-filter.c:648 648 if (prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, &preauth_program) == -1) { (gdb) step 654 else if (nnp_failed) { (gdb) step 662 pid_t pid_2 = getpid(); (gdb) stepi 0x008d62e4 in getpid at plt () (gdb) stepi 0x008d62e8 in getpid at plt () (gdb) stepi 0x008d62ec in getpid at plt () (gdb) stepi 0xf7c04800 in getpid () from /lib/arm-linux-gnueabihf/libc.so.6 (gdb) stepi 0xf7c04804 in getpid () from /lib/arm-linux-gnueabihf/libc.so.6 (gdb) disassemble Dump of assembler code for function getpid: 0xf7c04800 <+0>: push {r7} ; (str r7, [sp, #-4]!) => 0xf7c04804 <+4>: mov r7, #20 0xf7c04808 <+8>: svc 0x00000000 0xf7c0480c <+12>: pop {r7} ; (ldr r7, [sp], #4) 0xf7c04810 <+16>: bx lr End of assembler dump. (gdb) stepi 0xf7c04808 in getpid () from /lib/arm-linux-gnueabihf/libc.so.6 (gdb) disassemble Dump of assembler code for function getpid: 0xf7c04800 <+0>: push {r7} ; (str r7, [sp, #-4]!) 0xf7c04804 <+4>: mov r7, #20 => 0xf7c04808 <+8>: svc 0x00000000 0xf7c0480c <+12>: pop {r7} ; (ldr r7, [sp], #4) 0xf7c04810 <+16>: bx lr End of assembler dump. (gdb) stepi Program terminated with signal SIGSYS, Bad system call. The program no longer exists. See file `gdb-output.txt` for the full output. ### DUMP OF BPF FILTERS ### Before the call to `prctl(PR_SET_SECCOMP, ...)`, I added a print of `__NR_getpid` https://github.com/openssh/openssh-portable/blob/V_9_2_P1/sandbox-seccomp-filter.c#L438 #ifdef __NR_getpgid debug3("SC_ALLOW(__NR_getpgid=%d),", __NR_getpgid); #endif That code snippet is taken from the creation of the BPF filter defined at compile-time. https://github.com/openssh/openssh-portable/blob/V_9_2_P1/sandbox-seccomp-filter.c#L229 This prints `__NR_getpid=20` which means it is defined and BPF allowed. debug3: SC_ALLOW(__NR_getpid=20), [preauth] See file `bpf-filters-dumped.txt`. -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-31 02:58 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #30 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3779 --> https://bugzilla.mindrot.org/attachment.cgi?id=3779&action=edit gdp-output.txt gdb output before the errorring call to getpid -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at mindrot.org
2023-Dec-31 02:59 UTC
[Bug 3639] server thread aborts during client login after receiving SSH2_MSG_KEXINIT due to ssh_sandbox_violation
https://bugzilla.mindrot.org/show_bug.cgi?id=3639 --- Comment #31 from JM <jtm.moon.forum.user+mindrot at gmail.com> --- Created attachment 3780 --> https://bugzilla.mindrot.org/attachment.cgi?id=3780&action=edit bpf-filters-dumped.txt printing the BPF filters at runtime -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
Reasonably Related Threads
- [Bug 3702] New: sshd fork crashed when compiled with seccomp
- [Bug 1207] unsuccessful_login_count gets incremented by scp
- Annonymous access guest ok = yes
- [Bug 3714] New: sftp fails in one direction
- [Bug 3729] New: the new sshd does not work under the supervision of inetd