samba - Feb 2019 - winbind causing huge timeouts/delays since 4.8

> On 23 Feb 2019, at 14:07, Rowland Penny via samba <samba at
lists.samba.org> wrote:
> 
> On Sat, 23 Feb 2019 13:47:54 +0100
> Remy Zandwijk via samba <samba at lists.samba.org> wrote:
> 
>> 
>> 
>>> On 23 Feb 2019, at 09:33, Rowland Penny via samba
>>> <samba at lists.samba.org> wrote:
>>> 
>>> If you have, as you have, 'files sss winbind' in the the
passwd &
>>> group line in nsswitch.conf, means this:
>>> First /etc/passwd or /etc/group is searched and if the user or
>>> group is found, this info is returned.
>>> Next sssd will be asked, 'do you know this user or group ?'
if
>>> found, the info is returned.
>>> Finally winbind will be asked, 'do you know this user or group
?' if
>>> found, the info is returned.
>>> 
>>> Lets take a user called 'fred', this user is in AD. The
first search
>>> will return nothing, so sssd is asked, this 'asks' AD and
returns
>>> the users info. Finally, wait that's it, we have the info,
there is
>>> no need to ask winbind for anything.
>> 
>> That is incorrect. Alexander stated:
>> 
>>> No. we use max. 3 auth providers: (1. and 2. on all unix servers)
>>> 1. unix (local passwd)
>>> for static OS/service accounts across all our env
>>> 2. sssd (with unix ldap servers as provider)
>>> unix experienced user and application related service accounts
>>> 3. samba/winbind
>>> for windows users/services needing access to a group of unix
>>> servers
>> 
>> And:
>> 
>>> They don't - as stated above we use sssd for query/caching
entries
>>> from our ldap directory server and not Windows DomainConmtrollers -
>>> also this is possible, but makes more trouble and don't provide
>>> what samba's smb/windbind does.
>> 
>> He clearly writes (in multiple emails) that sssd is configured to use
>> his unix ldap servers and not AD.
>> 
>> Maybe three sources of user databases is not regular, but I fail to
>> see why this should be a problem (provided that usernames, uidNumbers
>> and such are unique across the databases).
> 
> And there is the problem, if 'fred' is in /etc/passwd, that user
will
> be used, but what if you meant fred in ldap or AD ?
> 
> There is absolutely no point in having 4 databases (yes there are 4,
> Unix, sssd, winbind and the ldap lines in smb.conf), they could all be
> combined in AD. 
> 
> The main problem is that the OP wants Samba changing to cope with his
> mess, it might be a valid change, but the reason for the change is
> invalid.
Well, I think the problem is you _assume_ users are in multiple databases and we
just don't know that. I think there is a good change Alexander perfectly
knows what he is doing and users are unique across databases.

Nevertheless, at some point nss is clearly querying winbind, which means nss did
not find the user in either /etc/passwd nor via sssd. In the case that winbind
_is_ queried, Alexander is experiencing, like he wrote, 'frequently system
hangs/slowness for a couple of seconds' and he observed that winbind is
causing this behaviour.

So maybe we should set our focus on winbind instead of the multiple database
stuff and figure out why it behaves like this since the upgrade from 4.7 to 4.8.
I would say we should start with fixing the winbind stuff in smb.conf. Right?

-Remy


P.S. I am following this thread since I also noticed occasional 'hangs'
when the system is querying winbind. This is Samba 4.8.7 on FreeBSD 11.2.

On Sat, 23 Feb 2019 14:21:57 +0100
Remy Zandwijk via samba <samba at lists.samba.org> wrote:
> 
> 
> > On 23 Feb 2019, at 14:07, Rowland Penny via samba
> > <samba at lists.samba.org> wrote:
> > 
> > On Sat, 23 Feb 2019 13:47:54 +0100
> > Remy Zandwijk via samba <samba at lists.samba.org> wrote:
> > 
> >> 
> >> 
> >>> On 23 Feb 2019, at 09:33, Rowland Penny via samba
> >>> <samba at lists.samba.org> wrote:
> >>> 
> >>> If you have, as you have, 'files sss winbind' in the
the passwd &
> >>> group line in nsswitch.conf, means this:
> >>> First /etc/passwd or /etc/group is searched and if the user or
> >>> group is found, this info is returned.
> >>> Next sssd will be asked, 'do you know this user or group
?' if
> >>> found, the info is returned.
> >>> Finally winbind will be asked, 'do you know this user or
group ?'
> >>> if found, the info is returned.
> >>> 
> >>> Lets take a user called 'fred', this user is in AD.
The first
> >>> search will return nothing, so sssd is asked, this
'asks' AD and
> >>> returns the users info. Finally, wait that's it, we have
the
> >>> info, there is no need to ask winbind for anything.
> >> 
> >> That is incorrect. Alexander stated:
> >> 
> >>> No. we use max. 3 auth providers: (1. and 2. on all unix
servers)
> >>> 1. unix (local passwd)
> >>> for static OS/service accounts across all our env
> >>> 2. sssd (with unix ldap servers as provider)
> >>> unix experienced user and application related service accounts
> >>> 3. samba/winbind
> >>> for windows users/services needing access to a group of unix
> >>> servers
> >> 
> >> And:
> >> 
> >>> They don't - as stated above we use sssd for query/caching
entries
> >>> from our ldap directory server and not Windows
DomainConmtrollers
> >>> - also this is possible, but makes more trouble and don't
provide
> >>> what samba's smb/windbind does.
> >> 
> >> He clearly writes (in multiple emails) that sssd is configured to
> >> use his unix ldap servers and not AD.
> >> 
> >> Maybe three sources of user databases is not regular, but I fail
to
> >> see why this should be a problem (provided that usernames,
> >> uidNumbers and such are unique across the databases).
> > 
> > And there is the problem, if 'fred' is in /etc/passwd, that
user
> > will be used, but what if you meant fred in ldap or AD ?
> > 
> > There is absolutely no point in having 4 databases (yes there are 4,
> > Unix, sssd, winbind and the ldap lines in smb.conf), they could all
> > be combined in AD. 
> > 
> > The main problem is that the OP wants Samba changing to cope with
> > his mess, it might be a valid change, but the reason for the change
> > is invalid.
> 
> Well, I think the problem is you _assume_ users are in multiple
> databases and we just don't know that. I think there is a good change
> Alexander perfectly knows what he is doing and users are unique
> across databases.
> 
> Nevertheless, at some point nss is clearly querying winbind, which
> means nss did not find the user in either /etc/passwd nor via sssd.
> In the case that winbind _is_ queried, Alexander is experiencing,
> like he wrote, 'frequently system hangs/slowness for a couple of
> seconds' and he observed that winbind is causing this behaviour.
> 
> So maybe we should set our focus on winbind instead of the multiple
> database stuff and figure out why it behaves like this since the
> upgrade from 4.7 to 4.8. I would say we should start with fixing the
> winbind stuff in smb.conf. Right?
> 
> -Remy
> 
> 
> P.S. I am following this thread since I also noticed occasional
> 'hangs' when the system is querying winbind. This is Samba 4.8.7 on
> FreeBSD 11.2.
> 
> 
> 
> 
I am quite prepared to help in getting winbind working correctly, but
this will require the OP changing their smb.conf considerably and
removing sssd. We do not support sssd, it is not a Samba product (for
want of a better name). Samba on a Unix domain member is designed
around 3 binaries, smbd, nmbd and winbind, the latter can do just about
anything sssd can do, so why use sssd ? Now you say that I am making
assumptions, well about this one, probably somewhere in the mix there
will be Windows domain members and the users in ldap are unlikely to
be known to them.

I have seen quite a few Samba setups that are like this one, bending
Samba to do something it isn't designed to do, you then get complaints
that it slow, hangs etc. Probably fixing the set up would stop all
these problems.

The OP says that it is sssd that is doing the ldap lookups, yet he has
these in smb.conf:

ldap connection timeout = 10
ldap timeout = 30

He also has these:

idmap config * : rangesize = 1000000
idmap config * : range = 1000000-19999999
idmap config * : backend = autorid

The '*' domain is meant for the Well Known SIDs and anything outside
the Samba domain. I would have expected something like this:

idmap config * : backend = tdb
idmap config * : range = 3000-7999
idmap config OPS : backend = rid
idmap config OPS : range = 10000-999999

If he wants to use Samba in a supported way, then I am more than
willing to help.

Rowland

> On 23 Feb 2019, at 14:52, Rowland Penny via samba <samba at
lists.samba.org> wrote:
> 
> On Sat, 23 Feb 2019 14:21:57 +0100
> Remy Zandwijk via samba <samba at lists.samba.org> wrote:
> 
>> 
>> 
>>> On 23 Feb 2019, at 14:07, Rowland Penny via samba
>>> <samba at lists.samba.org> wrote:
>>> 
>>> On Sat, 23 Feb 2019 13:47:54 +0100
>>> Remy Zandwijk via samba <samba at lists.samba.org> wrote:
>>> 
>>>> 
>>>> 
>>>>> On 23 Feb 2019, at 09:33, Rowland Penny via samba
>>>>> <samba at lists.samba.org> wrote:
>>>>> 
>>>>> If you have, as you have, 'files sss winbind' in
the the passwd &
>>>>> group line in nsswitch.conf, means this:
>>>>> First /etc/passwd or /etc/group is searched and if the user
or
>>>>> group is found, this info is returned.
>>>>> Next sssd will be asked, 'do you know this user or
group ?' if
>>>>> found, the info is returned.
>>>>> Finally winbind will be asked, 'do you know this user
or group ?'
>>>>> if found, the info is returned.
>>>>> 
>>>>> Lets take a user called 'fred', this user is in AD.
The first
>>>>> search will return nothing, so sssd is asked, this
'asks' AD and
>>>>> returns the users info. Finally, wait that's it, we
have the
>>>>> info, there is no need to ask winbind for anything.
>>>> 
>>>> That is incorrect. Alexander stated:
>>>> 
>>>>> No. we use max. 3 auth providers: (1. and 2. on all unix
servers)
>>>>> 1. unix (local passwd)
>>>>> for static OS/service accounts across all our env
>>>>> 2. sssd (with unix ldap servers as provider)
>>>>> unix experienced user and application related service
accounts
>>>>> 3. samba/winbind
>>>>> for windows users/services needing access to a group of
unix
>>>>> servers
>>>> 
>>>> And:
>>>> 
>>>>> They don't - as stated above we use sssd for
query/caching entries
>>>>> from our ldap directory server and not Windows
DomainConmtrollers
>>>>> - also this is possible, but makes more trouble and
don't provide
>>>>> what samba's smb/windbind does.
>>>> 
>>>> He clearly writes (in multiple emails) that sssd is configured
to
>>>> use his unix ldap servers and not AD.
>>>> 
>>>> Maybe three sources of user databases is not regular, but I
fail to
>>>> see why this should be a problem (provided that usernames,
>>>> uidNumbers and such are unique across the databases).
>>> 
>>> And there is the problem, if 'fred' is in /etc/passwd, that
user
>>> will be used, but what if you meant fred in ldap or AD ?
>>> 
>>> There is absolutely no point in having 4 databases (yes there are
4,
>>> Unix, sssd, winbind and the ldap lines in smb.conf), they could all
>>> be combined in AD. 
>>> 
>>> The main problem is that the OP wants Samba changing to cope with
>>> his mess, it might be a valid change, but the reason for the change
>>> is invalid.
>> 
>> Well, I think the problem is you _assume_ users are in multiple
>> databases and we just don't know that. I think there is a good
change
>> Alexander perfectly knows what he is doing and users are unique
>> across databases.
>> 
>> Nevertheless, at some point nss is clearly querying winbind, which
>> means nss did not find the user in either /etc/passwd nor via sssd.
>> In the case that winbind _is_ queried, Alexander is experiencing,
>> like he wrote, 'frequently system hangs/slowness for a couple of
>> seconds' and he observed that winbind is causing this behaviour.
>> 
>> So maybe we should set our focus on winbind instead of the multiple
>> database stuff and figure out why it behaves like this since the
>> upgrade from 4.7 to 4.8. I would say we should start with fixing the
>> winbind stuff in smb.conf. Right?
>> 
>> -Remy
>> 
>> 
>> P.S. I am following this thread since I also noticed occasional
>> 'hangs' when the system is querying winbind. This is Samba
4.8.7 on
>> FreeBSD 11.2.
>> 
>> 
>> 
>> 
> 
> I am quite prepared to help in getting winbind working correctly, but
> this will require the OP changing their smb.conf considerably and
> removing sssd. We do not support sssd, it is not a Samba product (for
> want of a better name). Samba on a Unix domain member is designed
> around 3 binaries, smbd, nmbd and winbind, the latter can do just about
> anything sssd can do, so why use sssd ? Now you say that I am making
> assumptions, well about this one, probably somewhere in the mix there
> will be Windows domain members and the users in ldap are unlikely to
> be known to them.
I consider sssd as 'just another' user database, like /etc/passwd (which
Samba apparently does support) and I personally cannot see any difference there,
but I respect your opinion.

Where is it documented winbind should be the only service which should be used
with nss? If it is not documented, maybe it should.

The proof of the pudding is of course, Alexander removing sssd from
nsswitch.conf and show us the problem still exists, or better yet, disappeared.

> I have seen quite a few Samba setups that are like this one, bending
> Samba to do something it isn't designed to do, you then get complaints
> that it slow, hangs etc. Probably fixing the set up would stop all
> these problems.
> 
> The OP says that it is sssd that is doing the ldap lookups, yet he has
> these in smb.conf:
> 
> ldap connection timeout = 10
> ldap timeout = 30
Yep, these lines should be removed.

> He also has these:
> 
> idmap config * : rangesize = 1000000
> idmap config * : range = 1000000-19999999
> idmap config * : backend = autorid
> 
> The '*' domain is meant for the Well Known SIDs and anything
outside
> the Samba domain. I would have expected something like this:
> 
> idmap config * : backend = tdb
> idmap config * : range = 3000-7999
> idmap config OPS : backend = rid
> idmap config OPS : range = 10000-999999
That should also be fixed.

> If he wants to use Samba in a supported way, then I am more than
> willing to help.
Thanks. Now let's hope Alexander is willing to jump some hoops.

-Remy