openssh bugs - May 2022 - [Bug 3430] New: 64 bit time and seccomp conflict

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

            Bug ID: 3430
           Summary: 64 bit time and seccomp conflict
           Product: Portable OpenSSH
           Version: 8.9p1
          Hardware: ARM
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: sshd
          Assignee: unassigned-bugs at mindrot.org
          Reporter: alacki93 at gmail.com

I found that glibc with support for 64-bit time could crash sshd
process with enabled seccomp seccomp. Test environment:
-Kernel 4.14.x
-32 bit ARM CPU
-glibc 2.34
-OpenSSH 8.9p1
-Toolchain: GCC 10

Syscall wrappers like a clock_gettime
(https://github.com/bminor/glibc/blob/glibc-2.34/sysdeps/unix/sysv/linux/clock_gettime.c)
first run syscall dedicated for kernels that support 64 bit time. If
this syscall failed with ENOSYS error, then glibc calls traditional
variant for 32 bit time.

OpenSSH in source code tries to figure which syscalls are supported by
kernel and only supported syscalls are added as an allowed in seccomp
(https://github.com/openssh/openssh-portable/blob/master/sandbox-seccomp-filter.c).
Because kernel 4.14 doesn?t support syscalls for 64 bit time,
__NR_clock_gettime64 is not added as a trusted syscall to seccomp. As a
result OpenSSH process is killed by seccomp every time when it tries to
use clock_gettime from glibc.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

Darren Tucker <dtucker at dtucker.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dtucker at dtucker.net

--- Comment #1 from Darren Tucker <dtucker at dtucker.net> ---
I don't follow your description of what's happening.  The sandbox code
has this (and has had since about 8.6p1, see bz#3093):

#ifdef __NR_clock_gettime64
        SC_ALLOW(__NR_clock_gettime64),
#endif

(In reply to Lacky from comment #0)
[...]
> OpenSSH in source code tries to figure which syscalls are supported
> by kernel and only supported syscalls are added as an allowed in
> seccomp
That's not an accurate description.  It allows any of the syscalls in
its list for which there is a definition, regardless of whether or not
it's supported by the currently running kernel.

If you build an sshd against and older set of headers that does not
define __NR_clock_gettime64 then it will not be included, but that's
because sshd has no way of knowing about it or what the syscall number
is at compile time.  I could imagine this biting you if you installed a
new libc.so without recompiling sshd with the new headers.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #2 from Lacky <alacki93 at gmail.com> ---
Created attachment 3592
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3592&action=edit
Log from sshd -D -ddd

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #3 from Lacky <alacki93 at gmail.com> ---
Created attachment 3593
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3593&action=edit
Log from strace -f

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #4 from Lacky <alacki93 at gmail.com> ---
(In reply to Darren Tucker from comment #1)
> 
> If you build an sshd against and older set of headers that does not
> define __NR_clock_gettime64 then it will not be included, but that's
> because sshd has no way of knowing about it or what the syscall
> number is at compile time.  I could imagine this biting you if you
> installed a new libc.so without recompiling sshd with the new
> headers.
Yes, but glibc doesn?t check it. They define syscalls in source code
(https://github.com/bminor/glibc/blob/glibc-2.34/sysdeps/unix/sysv/linux/arm/arch-syscall.h).

Now what happens on described test environment. Every time when sshd
tries to use clock_gettime from glibc you can find in strace something
like this: 

4628  clock_gettime64(CLOCK_BOOTTIME, 0xbe8c1488) = -1 ENOSYS (Function
not implemented) 
4628  clock_gettime(CLOCK_BOOTTIME, {tv_sec=247, tv_nsec=653301168}) 0 

Glibc first tries to use clock_gettime for 64 bit time. This fails
because kernel 4.14 doesn?t have support for this syscall (ENOSYS).
Then glibc tries to use classic clock_gettime. Usually this is not a
problem, but sshd uses seccomp. As you said sshd doesn?t add
clock_gettime64 as an allowed syscall to seccomp because it cannot find
it in kernel headers. Now let?s look at strace log. This entry means
that seccomp was initialized successfully for process with PID 4637: 

4637  prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, {len=107,
filter=0x4ee324} <unfinished ...> 
4637  <... prctl resumed>)              = 0 

After initialization of seccomp, process 4637 is killed when it tries
use unallowed clock_gettime64: 

4637  clock_gettime64(CLOCK_BOOTTIME,  <unfinished ...> 
4637  <... clock_gettime64 resumed> <unfinished ...>) = ? 
4637  +++ killed by SIGSYS +++ 
4628  --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=4637,
si_uid=101, si_status=SIGSYS, si_utime=4, si_stime=0} --- 
4628  kill(4637, SIGKILL) 
>From user perspective it looks like connection with server was closed
before any password prompt. 

In attachments you can find log from sshd (run with ?-D ?ddd") and
strace (run with ?-f?). 

Temporary solution to fix this bugs are: 
-Building OpenSSH without seccomp
-Building kernel without seccomp 
-Patching glibc to remove clock_gettime64 syscall. 

Disabling seccomp is the easiest workaround, but it decreases security
of sshd.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #5 from Darren Tucker <dtucker at dtucker.net> ---
(In reply to Lacky from comment #4)
> As you said sshd doesn?t add
> clock_gettime64 as an allowed syscall to seccomp because it cannot
> find it in kernel headers.
This is the root of the problem.  Why isn't __NR_clock_gettime64
defined?

I have a Debian 11 32bit ARM system here with glibc 2.31 and it has it:

$ uname -a
Linux tock 5.16.10-bone14 #1bullseye PREEMPT Tue Feb 22 00:07:39 UTC
2022 armv7l GNU/Linux

$ sudo dpkg -l | grep libc-bin
ii  libc-bin    2.31-13+deb11u3     armhf     GNU C Library: Binaries

$ grep -r __NR_clock_gettime64 /usr/include/*
[...]
/usr/include/asm-generic/unistd.h:#define __NR_clock_gettime64 403
/usr/include/asm-generic/unistd.h:__SYSCALL(__NR_clock_gettime64,
sys_clock_gettime)

I confirmed it's set in sshd (to 403) by adding a debug call.

What distro are you using?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #6 from Lacky <alacki93 at gmail.com> ---
(In reply to Darren Tucker from comment #5)
> This is the root of the problem.  Why isn't __NR_clock_gettime64
> defined?
> 
> I have a Debian 11 32bit ARM system here with glibc 2.31 and it has
> it:
> 
> $ uname -a
> Linux tock 5.16.10-bone14 #1bullseye PREEMPT Tue Feb 22 00:07:39 UTC
> 2022 armv7l GNU/Linux
> 
Support for 64-bit time in 32 bit platforms has been added in kernel
5.1
(https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.1-Year-2038-Syscalls).
Older kernels (like 4.14.x) that doesn't have this feature are still
supported by kernel developers.
> 
> What distro are you using?
This is Buildroot-based Linux (https://buildroot.org/).

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #7 from Darren Tucker <dtucker at dtucker.net> ---
(In reply to Lacky from comment #6)
[...]
> Support for 64-bit time in 32 bit platforms has been added in kernel
> 5.1
The kernel version you're running should be irrelevant.  If you were
running the same binary on a kernel that had 64bit time you'd have the
same problem because the headers don't have __NR_clock_gettime64 and
thus sshd didn't add it to the filter.
> > What distro are you using?
> 
> This is Buildroot-based Linux (https://buildroot.org/).
Sorry, I don't think there's anything OpenSSH could do differently. 
You've got a glibc that's making system calls that aren't in the
exported list of system calls.  I think your buildroot should be
defining __NR_clock_gettime64 if its glibc is going to call it.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

Tabor Kelly <taborkelly at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |taborkelly at gmail.com

--- Comment #8 from Tabor Kelly <taborkelly at gmail.com> ---
I ran into this on openssh 8.9p1 with glibc 2.35 on kernel 4.14 running
on 32 bit arm. I "fixed" it with a stupid patch in
sandbox-seccomp-filter.c, but I think this is actually a glibc bug.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3430

--- Comment #9 from Tabor Kelly <taborkelly at gmail.com> ---
Created attachment 3836
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3836&action=edit
My quick hack to get around what is probably a glibc bug

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.