samba - Feb 2019 - winbind causing huge timeouts/delays since 4.8

On Sun, 24 Feb 2019 18:28:43 +0100
Ralph Böhme <slow at samba.org> wrote:
> 
> Am 24.02.2019 um 16:42 schrieb Rowland Penny via samba
> <samba at lists.samba.org>:
> > On Sun, 24 Feb 2019 15:58:39 +0100 Ralph Böhme <slow at
samba.org>
> > wrote:
> >> Another thing that a customer has just been bitten by, was a
subtle
> >> bug in winbindd's idmap cache that resulted in all xid2sid
requests
> >> going through the idmap backend, iow winbindd issued LDAP
requests.
> >> With a few thousand users, things came to a grinding halt.
> >> 
> >> https://bugzilla.samba.org/show_bug.cgi?id=13802
> >> 
> >> Patch just landed upstream.
> > 
> > That is the bug I was referring to and probably (amongst all the
> > other cruft) what was causing the OP's problem.
> 
> Unlikely.
It is was I thought, but as the OP's setup is so convoluted, it is hard
to say.
> 
> > However, this has nothing to
> > do with using the 'ad' backend with Active Directory. We keep
> > dancing around this problem, saying things like 'we need to fix
> > this', we have been saying this since Samba 4 was released.
> 
> Which problem? Fix what? Been saying what?
There have been numerous discussions about the 'ad' backend over the
years and they have all gone nowhere. The 'ad' backend still works in
the same way as it did when Samba 4 was released and you still have to
store the next uidNumber & gidNumber outside AD if you use the Samba
tools.
> 
> > Windows Uses the SID-RID to identify the user and the domain it
> > comes from, surely we can find a way to do this for Samba, we are
> > half way there with the 'rid' backend.
> 
> I'm not really what "there" implies for you, but it seems
> idmap_autorid is eventually the backend that takes you "there".
:)
No it doesn't, at the moment, the only way to get the same ID on all
Unix machines (this includes DC's) is to use the 'ad' backend. You
think autorid is the way forward, well sorry, but in my opinion, it
isn't.

Rowland

Am 24.02.2019 um 18:48 schrieb Rowland Penny via samba <samba at
lists.samba.org>:
> On Sun, 24 Feb 2019 18:28:43 +0100 Ralph Böhme <slow at samba.org>
wrote:
>> Am 24.02.2019 um 16:42 schrieb Rowland Penny via samba <samba at
lists.samba.org>:
>>> On Sun, 24 Feb 2019 15:58:39 +0100 Ralph Böhme <slow at
samba.org> wrote:
>>>> Another thing that a customer has just been bitten by, was a
subtle
>>>> bug in winbindd's idmap cache that resulted in all xid2sid
requests
>>>> going through the idmap backend, iow winbindd issued LDAP
requests.
>>>> With a few thousand users, things came to a grinding halt.
>>>> 
>>>> https://bugzilla.samba.org/show_bug.cgi?id=13802
>>>> 
>>>> Patch just landed upstream.
>>> 
>>> That is the bug I was referring to and probably (amongst all the
>>> other cruft) what was causing the OP's problem.
>> 
>> Unlikely.
> 
> It is was I thought, but as the OP's setup is so convoluted, it is hard
> to say.
I don't think it's convoluted, it's certainly beyond the simple
standard setup we all wish everybody was using, but I don't think it is
broken as is. I just think an appropriate analysis requires more resources then
is available on the list.
>>> However, this has nothing to
>>> do with using the 'ad' backend with Active Directory. We
keep
>>> dancing around this problem, saying things like 'we need to fix
>>> this', we have been saying this since Samba 4 was released.
>> 
>> Which problem? Fix what? Been saying what?
> 
> There have been numerous discussions about the 'ad' backend over
the
> years and they have all gone nowhere. The 'ad' backend still works
in
> the same way as it did when Samba 4 was released and you still have to
> store the next uidNumber & gidNumber outside AD if you use the Samba
> tools.
Looks like you're mixing AD DC use case with member server use case. Can we
please keep that seperate? Afaict, the one has nothing to do with the other.
>>> Windows Uses the SID-RID to identify the user and the domain it
>>> comes from, surely we can find a way to do this for Samba, we are
>>> half way there with the 'rid' backend.
>> 
>> I'm not really what "there" implies for you, but it seems
>> idmap_autorid is eventually the backend that takes you
"there". :)
> 
> No it doesn't, at the moment, the only way to get the same ID on all
> Unix machines (this includes DC's) is to use the 'ad' backend.
Sure. But only certain use cases require the same id on all machines, many
don't. I'm just saying that you should better not use idmap_ad, but
instead use eg idmap_autorid unless you're setup requires idmap_ad.
> You think autorid is the way forward, well sorry, but in my opinion, it
> isn't.
Rowland, this is not about *the* way forward, this is about using the right
backend at the right time.

-slow

On Sun, 24 Feb 2019 19:25:14 +0100
Ralph Böhme <slow at samba.org> wrote:
> 
> Am 24.02.2019 um 18:48 schrieb Rowland Penny via samba
> <samba at lists.samba.org>:
> > On Sun, 24 Feb 2019 18:28:43 +0100 Ralph Böhme <slow at
samba.org>
> > wrote:
> >> Am 24.02.2019 um 16:42 schrieb Rowland Penny via samba
> >> <samba at lists.samba.org>:
> >>> On Sun, 24 Feb 2019 15:58:39 +0100 Ralph Böhme <slow at
samba.org>
> >>> wrote:
> >>>> Another thing that a customer has just been bitten by, was
a
> >>>> subtle bug in winbindd's idmap cache that resulted in
all
> >>>> xid2sid requests going through the idmap backend, iow
winbindd
> >>>> issued LDAP requests. With a few thousand users, things
came to
> >>>> a grinding halt.
> >>>> 
> >>>> https://bugzilla.samba.org/show_bug.cgi?id=13802
> >>>> 
> >>>> Patch just landed upstream.
> >>> 
> >>> That is the bug I was referring to and probably (amongst all
the
> >>> other cruft) what was causing the OP's problem.
> >> 
> >> Unlikely.
> > 
> > It is was I thought, but as the OP's setup is so convoluted, it is
> > hard to say.
> 
> I don't think it's convoluted, it's certainly beyond the simple
> standard setup we all wish everybody was using, but I don't think it
> is broken as is. I just think an appropriate analysis requires more
> resources then is available on the list.
> 
> >>> However, this has nothing to
> >>> do with using the 'ad' backend with Active Directory.
We keep
> >>> dancing around this problem, saying things like 'we need
to fix
> >>> this', we have been saying this since Samba 4 was
released.
> >> 
> >> Which problem? Fix what? Been saying what?
> > 
> > There have been numerous discussions about the 'ad' backend
over the
> > years and they have all gone nowhere. The 'ad' backend still
works
> > in the same way as it did when Samba 4 was released and you still
> > have to store the next uidNumber & gidNumber outside AD if you use
> > the Samba tools.
> 
> Looks like you're mixing AD DC use case with member server use case.
> Can we please keep that seperate? Afaict, the one has nothing to do
> with the other.
So a Samba AD DC has nothing to do with a Unix member server, could
have fooled me ;-)

This is the sort of thinking that is holding Samba back. You use any
Windows domain computer and you get the same identity, You cannot do
this on Unix domain computers.
> 
> >>> Windows Uses the SID-RID to identify the user and the domain
it
> >>> comes from, surely we can find a way to do this for Samba, we
are
> >>> half way there with the 'rid' backend.
> >> 
> >> I'm not really what "there" implies for you, but it
seems
> >> idmap_autorid is eventually the backend that takes you
"there". :)
> > 
> > No it doesn't, at the moment, the only way to get the same ID on
all
> > Unix machines (this includes DC's) is to use the 'ad'
backend.
> 
> Sure. But only certain use cases require the same id on all machines,
> many don't. I'm just saying that you should better not use
idmap_ad,
> but instead use eg idmap_autorid unless you're setup requires
> idmap_ad.
I am not saying don't use autorid, I am saying that I will not use it,
I just do not see the point to it, the 'ad' and rid' backends work
for
most users. Perhaps if there was more documentation, I might change my
mind, but it took me long enough to understand the 'ad' backend, but
once I did, it all clicked in to place. The other problem is that 'ad'
is the only one that allows you to set individual login shells and Unic
home dirs.

Perhaps that is part of the 'problem', there are just too many winbind
backends.
> 
> > You think autorid is the way forward, well sorry, but in my
> > opinion, it isn't.
> 
> Rowland, this is not about *the* way forward, this is about using the
> right backend at the right time.
No Ralph, it is about *the* way forward, Samba needs to get to the
point that it works exactly like Windows (or better), Samba has to
outdo Windows.

Rowland

On 24.02.2019 19:25, Ralph Böhme via samba wrote:
> Am 24.02.2019 um 18:48 schrieb Rowland Penny via samba <samba at
lists.samba.org>:
>> On Sun, 24 Feb 2019 18:28:43 +0100 Ralph Böhme <slow at
samba.org> wrote:
>>> Am 24.02.2019 um 16:42 schrieb Rowland Penny via samba <samba at
lists.samba.org>:
>>>> On Sun, 24 Feb 2019 15:58:39 +0100 Ralph Böhme <slow at
samba.org> wrote:
>>>>> Another thing that a customer has just been bitten by, was
a subtle
>>>>> bug in winbindd's idmap cache that resulted in all
xid2sid requests
>>>>> going through the idmap backend, iow winbindd issued LDAP
requests.
>>>>> With a few thousand users, things came to a grinding halt.
>>>>>
>>>>> https://bugzilla.samba.org/show_bug.cgi?id=13802
>>>>>
>>>>> Patch just landed upstream.
>>>> That is the bug I was referring to and probably (amongst all
the
>>>> other cruft) what was causing the OP's problem.
>>> Unlikely.
>> It is was I thought, but as the OP's setup is so convoluted, it is
hard
>> to say.
> I don't think it's convoluted, it's certainly beyond the simple
standard setup we all wish everybody was using, but I don't think it is
broken as is. I just think an appropriate analysis requires more resources then
is available on the list.
>
>>>> However, this has nothing to
>>>> do with using the 'ad' backend with Active Directory.
We keep
>>>> dancing around this problem, saying things like 'we need to
fix
>>>> this', we have been saying this since Samba 4 was released.
>>> Which problem? Fix what? Been saying what?
>> There have been numerous discussions about the 'ad' backend
over the
>> years and they have all gone nowhere. The 'ad' backend still
works in
>> the same way as it did when Samba 4 was released and you still have to
>> store the next uidNumber & gidNumber outside AD if you use the
Samba
>> tools.
> Looks like you're mixing AD DC use case with member server use case.
Can we please keep that seperate? Afaict, the one has nothing to do with the
other.
I'm confused.. how is the choice of the idmap backend related to an AD 
DC use case?

>>>> Windows Uses the SID-RID to identify the user and the domain it
>>>> comes from, surely we can find a way to do this for Samba, we
are
>>>> half way there with the 'rid' backend.
>>> I'm not really what "there" implies for you, but it
seems
>>> idmap_autorid is eventually the backend that takes you
"there". :)
>> No it doesn't, at the moment, the only way to get the same ID on
all
>> Unix machines (this includes DC's) is to use the 'ad'
backend.
> Sure. But only certain use cases require the same id on all machines, many
don't. I'm just saying that you should better not use idmap_ad, but
instead use eg idmap_autorid unless you're setup requires idmap_ad.
Would you, or someone else mind sharing some of these use cases when 
idmap_ad would be necessary and when idmap_autorid would suffice? 
Specifically, in which situations do I absolutely need the ID to be the 
same on each member, and in which cases could I actually go without 
this? For example, if my AD is managed by Samba only but I only have 
Windows users who will never have to log in to a unix box, are there 
still advantages of the ad backend over the (auto)rid one?

I assume that most readers of the wiki will, like me, find that "central 
administration of ID's inside the AD" and "ID's not stored in
a local
database that can corrupt with lost file ownership" seem like really 
important arguments (btw, the last point is not stated as a disadvantage 
for the rid/autorid backend in the wiki). Reading this, it just seems 
that the ad backend is always the right one except that it's a headache 
to manage.

To put it differently, if Samba was improved in such a way that we could 
use the ad backend without having to manually manage the rfc2307 
attributes, wouldn't this be the best if not only solution we needed?

Viktor