openssh bugs - May 2025 - [Bug 3822] New: --with-linux-memlock-onfault causes segfaults with some combinations of PAM modules

https://bugzilla.mindrot.org/show_bug.cgi?id=3822

            Bug ID: 3822
           Summary: --with-linux-memlock-onfault causes segfaults with
                    some combinations of PAM modules
           Product: Portable OpenSSH
           Version: 10.0p2
          Hardware: Other
               URL: https://bugs.debian.org/1103418
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: PAM support
          Assignee: unassigned-bugs at mindrot.org
          Reporter: cjwatson at debian.org

I noticed the new --with-linux-memlock-onfault option when upgrading
Debian to 10.0, and it seemed like a good idea so I enabled it. 
However, shortly afterwards I got a report of somewhat random but
frequent segfaults in sshd-session (https://bugs.debian.org/1103418). 
It took me a while to track this down to establish that it wasn't the
fault of a Debian patch; in fact it turns out to be reproducible with
an unmodified OpenSSH portable tree, provided that I've configured
--with-pam and --with-linux-memlock-onfault, and that I have "enough"
PAM modules enabled.  The segfaults are generally somewhere under
pam_authenticate when a PAM module tries to allocate memory.  They
usually look something like this (there are stack traces with more
detail in the linked Debian bug report - this is just for
illustration):

  __printf_buffer (libc.so.6 + 0x6261d)
  __vasprintf_internal (libc.so.6 + 0x87a6b)
  ___asprintf_chk (libc.so.6 + 0x11abef)
  n/a (pam_ecryptfs.so + 0x23d7)
  pam_sm_authenticate (pam_ecryptfs.so + 0x2ace)
  n/a (libpam.so.0 + 0x44de)
  pam_authenticate (libpam.so.0 + 0x3be3)
  n/a (/home/cjwatson/openssh/sshd-session + 0x3dbf0)

But the code in pam_ecryptfs looks innocent.  And in mlockall(2), I see
this paragraph which looks relevant:

       If MCL_FUTURE has been specified, then a later system call
(e.g.,
       mmap(2), sbrk(2), malloc(3)), may fail if it would cause the
number of
       locked bytes to exceed the permitted maximum (see below).  In
the same
       circumstances, stack growth may likewise fail: the kernel will
deny
       stack expansion and deliver a SIGSEGV signal to the process.

I think this is probably because RLIMIT_MEMLOCK is typically not all
that huge (8 MiB by default on Debian), and PAM modules, the libraries
they depend on, and whatever memory they allocate on the stack might
well exceed that.  Commenting out the call to memlock_onfault_setup in
platform_pre_session_start makes the segfaults go away.  I tried just
disabling MCL_FUTURE for the sshd-session process, but that didn't seem
to help.

Perhaps it would make sense to lock memory in the session process only
once the session has been established (do_authenticated or server_loop2
or so)?  By that point the session process is just processing and
forwarding packets and shouldn't be doing anything like as much memory
allocation as it does while establishing a connection.  It's perhaps
not quite as good for the stated rationale of
6c49e5f7dcaf886b4a702a6c003cae9dca04d3ea (always being able to connect
even while the system is under heavy kcompactd load), but it should
still help with established connections.

For now I'm just going to disable --with-linux-memlock-onfault, but I
thought I should let you know in case you have any other ideas.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3822

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org

--- Comment #1 from Damien Miller <djm at mindrot.org> ---
IMO the places where this is most necessary are 1) the listener process
(i.e. sshd) and 2) the authentication phase, as both of these are the
most time-sensitive steps of the connection.

Maybe the compile-time could be made more granular to allow it to apply
to the listener only, or the listener+auth phase or everything.

I think it's always going to be possible to create some configuration
of PAM modules or session runtime behaviour (e.g. opening a gazillion
forwardings) that blows over the memlock limit, so if I were in your
shoes of having to pick a build configuration for everyone, then I'd
have it apply only to the listener.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3822

--- Comment #2 from Damien Miller <djm at mindrot.org> ---
(but even then, I bet I could make a giant sshd_config that would
exceed the memlock limit for the listener too...)

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3822

--- Comment #3 from Damien Miller <djm at mindrot.org> ---
Created attachment 3875
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3875&action=edit
--with-linux-memlock-onfault=listener

This is kind of what I mean: --with-linux-memlock-onfault=listener to
allow mlock()ing of the listener process only. This at least should
guarantee a SSH connection to _start_ when this kcompact thing is
causing pressure

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3822

--- Comment #4 from Colin Watson <cjwatson at debian.org> ---
Yes, I could go with that.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.