openssh unix dev - Jul 2018 - root can login to console but not via ssh

Hi,

I built OpenSSH 7.7p1-1 to try to include some security fixes for an old OS
version (SLES 10). We use a special PAM module for root to allow us to
provide auto-expiring passwords. There is, however, one root password that
should always work. root can login just fine on the console, which I assume
means that the PAM module is working correctly because I can use both the
always should work password and an auto-expiring password. And if I provide
a valid key in authorized_keys I can login via ssh without a password
without any problems.

I can also login as root just fine via ssh prior to installing the RPM I
built for OpenSSH 7.7p1-1. However, once I install it, I can no longer ssh
as root. I saved the file: /etc/pam.d/sshd from prior to the install and
restore it after the RPM is installed since it overwrites it. I have a
/etc/pam.d/common-auth that has:

test10:/etc/pam.d # cat common-auth
#
# /etc/pam.d/common-auth - authentication settings common to all services
#
# This file is included from other service-specific PAM config files,
# and should contain a list of the authentication modules that define
# the central authentication scheme for use on the system
# (e.g., /etc/shadow, LDAP, Kerberos, etc.).  The default is to use the
# traditional Unix authentication mechanisms.
#
auth required pam_env.so
auth sufficient pam_rootexpirepass.so
auth required pam_unix2.so

The /etc/pam.d/sshd has:

test10:/etc/pam.d # cat sshd
#%PAM-1.0
auth     include        common-auth
auth     required       pam_nologin.so
account  include        common-account
password include        common-password
session  include        common-session
# Enable the following line to get resmgr support for
# ssh sessions (see /usr/share/doc/packages/resmgr/README)
#session  optional pam_resmgr.so fake_ttyname

If I run sshd with debugging I see:

debug1: Server will not fork when running in debugging mode.
debug1: rexec start in 4 out 4 newsock 4 pipe -1 sock 7
debug1: inetd sockets after dupping: 3, 3
Connection from 10.10.0.34 port 38327 on 10.10.0.38 port 22
debug1: Client protocol version 2.0; client software version OpenSSH_4.2
debug1: match: OpenSSH_4.2 pat OpenSSH_2*,OpenSSH_3*,OpenSSH_4* compat
0x00000000
debug1: Local version string SSH-2.0-OpenSSH_7.7
debug1: permanently_set_uid: 71/65 [preauth]
debug1: list_hostkey_types:
ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
debug1: SSH2_MSG_KEXINIT sent [preauth]
debug1: SSH2_MSG_KEXINIT received [preauth]
debug1: kex: algorithm: diffie-hellman-group14-sha1 [preauth]
debug1: kex: host key algorithm: ssh-rsa [preauth]
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha1 compression:
none [preauth]
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha1 compression:
none [preauth]
debug1: expecting SSH2_MSG_KEXDH_INIT [preauth]
debug1: rekey after 4294967296 blocks [preauth]
debug1: SSH2_MSG_NEWKEYS sent [preauth]
debug1: expecting SSH2_MSG_NEWKEYS [preauth]
debug1: SSH2_MSG_NEWKEYS received [preauth]
debug1: rekey after 4294967296 blocks [preauth]
debug1: KEX done [preauth]
debug1: userauth-request for user root service ssh-connection method none
[preauth]
debug1: attempt 0 failures 0 [preauth]
debug1: PAM: initializing for "root"
debug1: PAM: setting PAM_RHOST to "10.10.0.34"
debug1: PAM: setting PAM_TTY to "ssh"
debug1: userauth-request for user root service ssh-connection method
keyboard-interactive [preauth]
debug1: attempt 1 failures 0 [preauth]
debug1: keyboard-interactive devs  [preauth]
debug1: auth2_challenge: user=root devs= [preauth]
debug1: kbdint_alloc: devices 'pam' [preauth]
debug1: auth2_challenge_start: trying authentication method 'pam'
[preauth]
Postponed keyboard-interactive for root from 10.10.0.34 port 38327 ssh2
[preauth]
PAM: Authentication failure for root from 10.10.0.34
Failed keyboard-interactive/pam for root from 10.10.0.34 port 38327 ssh2
debug1: Unable to open the btmp file /var/log/btmp: No such file or
directory
debug1: userauth-request for user root service ssh-connection method
keyboard-interactive [preauth]
debug1: attempt 2 failures 1 [preauth]
debug1: keyboard-interactive devs  [preauth]
debug1: auth2_challenge: user=root devs= [preauth]
debug1: kbdint_alloc: devices 'pam' [preauth]
debug1: auth2_challenge_start: trying authentication method 'pam'
[preauth]
Postponed keyboard-interactive for root from 10.10.0.34 port 38327 ssh2
[preauth]
Connection closed by authenticating user root 10.10.0.34 port 38327
[preauth]
debug1: do_cleanup [preauth]
debug1: do_cleanup
debug1: PAM: cleanup
debug1: Killing privsep child 2423

Any suggestions as to where I can look to determine why ssh is not able to
authenticate via PAM?

Thanks,

Rob

On Tue, 17 Jul 2018, Rob Marshall wrote:
> Hi,
> 
> I built OpenSSH 7.7p1-1 to try to include some security fixes for an old OS
> version (SLES 10). We use a special PAM module for root to allow us to
> provide auto-expiring passwords. There is, however, one root password that
> should always work. root can login just fine on the console, which I assume
> means that the PAM module is working correctly because I can use both the
> always should work password and an auto-expiring password. And if I provide
> a valid key in authorized_keys I can login via ssh without a password
> without any problems.
> 
> I can also login as root just fine via ssh prior to installing the RPM I
> built for OpenSSH 7.7p1-1. However, once I install it, I can no longer ssh
> as root. I saved the file: /etc/pam.d/sshd from prior to the install and
> restore it after the RPM is installed since it overwrites it. I have a
> /etc/pam.d/common-auth that has:
> test10:/etc/pam.d # cat sshd
> #%PAM-1.0
> auth     include        common-auth
> auth     required       pam_nologin.so
I think pam_nologin.so should be in the "account" rather than
"auth" stack.

I.e.

account	required	pam_nologin.so

-d

On Tue, Jul 17, 2018 at 6:22 PM, Damien Miller <djm at mindrot.org>
wrote:
>
>
> On Tue, 17 Jul 2018, Rob Marshall wrote:
>
>> Hi,
>>
>> I built OpenSSH 7.7p1-1 to try to include some security fixes for an
old OS
>> version (SLES 10). We use a special PAM module for root to allow us to
>> provide auto-expiring passwords. There is, however, one root password
that
>> should always work. root can login just fine on the console, which I
assume
>> means that the PAM module is working correctly because I can use both
the
>> always should work password and an auto-expiring password. And if I
provide
>> a valid key in authorized_keys I can login via ssh without a password
>> without any problems.
>>
>> I can also login as root just fine via ssh prior to installing the RPM
I
>> built for OpenSSH 7.7p1-1. However, once I install it, I can no longer
ssh
>> as root. I saved the file: /etc/pam.d/sshd from prior to the install
and
>> restore it after the RPM is installed since it overwrites it. I have a
>> /etc/pam.d/common-auth that has:
>> test10:/etc/pam.d # cat sshd
>
>> #%PAM-1.0
>> auth     include        common-auth
>> auth     required       pam_nologin.so
>
> I think pam_nologin.so should be in the "account" rather than
"auth" stack.
>
> I.e.
>
> account required        pam_nologin.so
>
> -d
Definitely check /etc/ssh/sshd_login, or wherever your particular
version of SSH expects its sshd_config, for the value of
"PermitRootLogin". If you're a weasel, and want to test SSHD
configs.

And.... This is why PAM is often a solution in search of a problem to
cause. It provides enormous flexibility, but has no effective
line-by-line management or review tool to double check the frequently
confusing results of manual editing. And there are a *lot* of
Google-discoverable manual tune-ups that break things you don't expect
at times you can't afford. The few PAM management tools (such as
authconfig) do not keep PAM consistent with previous manual edits. The
results can be seriously destructive.

If you'd like to review what it was before any manual auditing or
before "authconfig" was run to tune your local environment,  I urge
you to do "rpm -q -f /etc/pam.d/sshd", get the original RPM that
provided the original file, and take it apart with "rpm2cpio.sh
filename | cpio -id" to see what the original file looked like, then
compare it side-by-side for the results of editing.