openssh unix dev - Jun 2002 - OpenSSH and Solaris groups

I have an odd problem and I was wondering if anyone has ever run into this
before.  I have a machine running solaris 8, OpenSSH 3.1p1 and OpenSSL
0.9.6c and it has been working fine for quite some time (ssh that is).
Today, /etc/system was updated to increase the maximum number of groups
from 16 to 32.  After the system was rebooted, things seemed to be working
as expected, however one of our users who is a member of 21 groups (don't
ask) is now unable to log in.  Previously the extra groups over 16 were
ignored and things were fine, but it seems like now that the system is
recognizing membership to these groups ssh is failing.  The error that
appears in the log file and when connecting to a port running a debugging
server reports that getgroups failed with an invalid argument.  This is
happening from all machines... except one which is the strange thing.  If
the user connects via ssh from one certain machine, the error still
appears in the log file, however the login is successful.  Has anyone ever
run into something like this before?  Any and all advice would be greatly
appreciated.  Thanks.

sshd[847]: [ID 800047 auth.crit] fatal: getgroups: Invalid argument

Matt Studley
American Mathematical Society
UNIX Sys Admin        		  "Quantum Mechanics -
mjs at ams.org			  	The dreams that stuff is made of"

NGROUPS_MAX is 16 in /usr/include/limits.h.  OpenSSH uses this in it's
call to getgroups() so EINVAL is the proper error for it to return
according to "man getgroups".  I have no idea why it works for one
machine.  You might try changing NGROUPS_MAX to MY_NGROUPS_MAX in the
code, and set it to 32 or some larger number.

By the way, how did you change the maximum number of groups in
/etc/system?  Is this a "documented" feature?  It seems like a good
number
of things would break... scary... ;-)  I feel your pain tho'... we have at
least one user with a list of 40+ groups that fluctuates and we
occasionally get complaints that the ones he wants to have aren't in the
16 he gets... ;-)

Thanks,

	Ed

On Tue, 18 Jun 2002, Matt Studley wrote:
> Date: Tue, 18 Jun 2002 14:24:42 -0400 (EDT)
> From: Matt Studley <mjs at ams.org>
> To: openssh-unix-dev at mindrot.org
> Subject: OpenSSH and Solaris groups
>
>
> I have an odd problem and I was wondering if anyone has ever run into this
> before.  I have a machine running solaris 8, OpenSSH 3.1p1 and OpenSSL
> 0.9.6c and it has been working fine for quite some time (ssh that is).
> Today, /etc/system was updated to increase the maximum number of groups
> from 16 to 32.  After the system was rebooted, things seemed to be working
> as expected, however one of our users who is a member of 21 groups
(don't
> ask) is now unable to log in.  Previously the extra groups over 16 were
> ignored and things were fine, but it seems like now that the system is
> recognizing membership to these groups ssh is failing.  The error that
> appears in the log file and when connecting to a port running a debugging
> server reports that getgroups failed with an invalid argument.  This is
> happening from all machines... except one which is the strange thing.  If
> the user connects via ssh from one certain machine, the error still
> appears in the log file, however the login is successful.  Has anyone ever
> run into something like this before?  Any and all advice would be greatly
> appreciated.  Thanks.
>
> sshd[847]: [ID 800047 auth.crit] fatal: getgroups: Invalid argument
>
> Matt Studley
> American Mathematical Society
> UNIX Sys Admin        		  "Quantum Mechanics -
> mjs at ams.org			  	The dreams that stuff is made of"
>
> _______________________________________________
> openssh-unix-dev at mindrot.org mailing list
> http://www.mindrot.org/mailman/listinfo/openssh-unix-dev
>
Ed Phillips <ed at udel.edu> University of Delaware (302) 831-6082
Systems Programmer III, Network and Systems Services
finger -l ed at polycut.nss.udel.edu for PGP public key

On Tue, Jun 18, 2002 at 02:24:42PM -0400, Matt Studley
wrote:
> I have an odd problem and I was wondering if anyone has ever run into this
> before.  I have a machine running solaris 8, OpenSSH 3.1p1 and OpenSSL
> 0.9.6c and it has been working fine for quite some time (ssh that is).
> Today, /etc/system was updated to increase the maximum number of groups
> from 16 to 32.  After the system was rebooted, things seemed to be working
> as expected, however one of our users who is a member of 21 groups
(don't
> ask) is now unable to log in.  Previously the extra groups over 16 were
> ignored and things were fine, but it seems like now that the system is
> recognizing membership to these groups ssh is failing.  The error that
> appears in the log file and when connecting to a port running a debugging
> server reports that getgroups failed with an invalid argument.  This is
> happening from all machines... except one which is the strange thing.  If
> the user connects via ssh from one certain machine, the error still
> appears in the log file, however the login is successful.  Has anyone ever
> run into something like this before?  Any and all advice would be greatly
> appreciated.  Thanks.
it can fail in the client or the server, though i'm less familiar
with the UID swapping on the server side.  if it's the client, you
might try to remove set-uid bit.  i don't remember the version that
first had supplementary group handling in uidswap.

we should perhaps use sysconf to get the run-time value.  what is
the system tunable for ngroups on solaris?

should this work:

[stevesk at scott stevesk]$ getconf _SC_NGROUPS_MAX
getconf: Invalid argument (_SC_NGROUPS_MAX)

[stevesk at scott stevesk]$ getconf -a|grep NGROUPS
NGROUPS_MAX:                    16
_POSIX_NGROUPS_MAX:             0