samba - Dec 2021 - Get last uidNumber

On Sun, 2021-02-14 at 22:27 -0500, Jonathon Reinhart
wrote:
> On Sun, Feb 14, 2021 at 6:48 PM Andrew Bartlett <abartlet at
samba.org>
> wrote:
> > On Tue, 2020-11-03 at 11:38 -0500, Jonathon Reinhart via samba
> > wrote:
> > > See this post where Rowland suggested using these attributes to
> > > store
> > > the maximum value in LDAP. (These are not automatically
> > > calculated,
> > > your script would need to keep them updated.)
> > > 
> > > - msSFU30MaxUidNumber
> > > - msSFU30MaxGidNumber
> > 
> > Just be aware that these are not multi-master update safe to
> > update.
> > 
> > If your domain can get out of sync, just be aware that if your tool
> > is
> > pointed at an 'older' DC, it could allocate the same UID
twice.
> 
> Thanks for the heads up. Am I correct that this is problematic even
> without the atomic update problem you mention below?
Yes, the atomic update only helps on the single DC.  The joys of multi-
master replication.
> Let's assume that msSFU30MaxUidNumber is being updated atomically
> (from the perspective of the LDAP server serving adman's requests).
> If
> I understand your concern correctly, the following sequence could
> happen:
> 
> - msSFU30MaxUidNumber = 1000
> - Adman connects to DC1, assigns UID=1000 to a user, and atomically
> increments msSFU30MaxUidNumber to 1001
> - Before the replication completes, Adman connects to DC2, sees the
> old value of msSFU30MaxUidNumber = 1000, and assigns a duplicate
> UID=1000 to a different user
> - msSFU30MaxUidNumber = 1001
Exactly.
> I have a cron job that runs Adman every minute. Since Adman looks up
> the LDAP server via the SRV record [1] (and doesn't sort them),
it's
> possible that it could choose a different DC the next time. And I
> guess if replication was being pokey, this scenario could happen.
> 
> I'm not sure how to best resolve this. Perhaps one mitigating method
> would be to always target the DC with the PDC emulator FSMO role.
> I've opened issue adman#23 [2] to track this.
I recommend RID based allocation, always take the user RID and add a
base.  Ideally record that base in the trustPosixOffset attribute.
> > > https://lists.samba.org/archive/samba/2019-June/223499.html
> > > 
> > > But also, check out my project ADMan, which does several things,
> > > one
> > > of which is assign uidNumber / gidNumber attributes to new users:
> > > 
> > > https://gitlab.com/JonathonReinhart/adman/
> > 
> > Cool.
> > 
> > One thing I would love is to get the uidNumber/gidNumber assignment
> > into Samba.  I would use something like the algorithm from
> > idmap_sssd
> > to pick a 'pretty much likely to be unique UID' based on the
RID
> > and
> > then store it forever in the uidNumber.
> 
> One mistake I would love to go back in time to rectify is the
> suggested UID/GID ranges. I first asked about this here [3]. I'm
> currently using 100000-200000 for both uidNumber and gidNumber. While
> this is fine for most use cases, it does preclude the use of SSSD's
> "automatic user private groups" feature which, if enabled
> (auto_private_groups = true), makes it such that "every AD user has a
> GID which is identical to the UID." [4] [5] For that reason, I wish I
> had gone with disjoint ranges.
Well, if you move to RID based, as users and group share a RID space,
this becomes available again (for new users).  I agree, keeping UIDs
and GIDs as a single number space is preferable. 
> > In the meantime I would suggest to try to do an LDAP atomic update
> > of
> > the msSFU30MaxUidNumber regardless.
> > See https://ldapwiki.com/wiki/LDIF%20Atomic%20Operations for what I
> > mean.
> 
> Cool. I had briefly considered the atomicity problems, but kind of
> ignored them, as Adman is the only agent in my network updating these
> fields. But this would be a great improvement. I actually identified
> another related problem while looking at this. I opened adman#24 [6]
> to track.
> 
> > Sadly Samba as an ideal POSIX directory remains on my 'todo'
list.
> > 
> > Some ideas are here:
> > https://wiki.samba.org/index.php/Better_Posix_AD
> 
> Yep. If people are running Samba AD DCs, they're almost certainly
> running POSIX clients/servers too. If Samba managed some/all of this
> automatically, it would save a lot of start-up hassle.
I would love someone to make a start on this :-).
> Not having to know intimate details (like Rowland's favorite: Don't
> assign gidNumber to Domain Admins b/c it owns things in SYSVOL) would
> save a lot of mailing list traffic.
> 
> Thanks for taking a look and offering insight!
You are welcome,

Andrew Bartlett
-- 
Andrew Bartlett (he/him)       https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead, Catalyst IT   https://catalyst.net.nz/services/samba

On Sun, Feb 14, 2021 at 10:50 PM Andrew Bartlett <abartlet at samba.org>
wrote:
>
> On Sun, 2021-02-14 at 22:27 -0500, Jonathon Reinhart wrote:
> > On Sun, Feb 14, 2021 at 6:48 PM Andrew Bartlett <abartlet at
samba.org>
> > wrote:
> > > On Tue, 2020-11-03 at 11:38 -0500, Jonathon Reinhart via samba
> > > wrote:
> > > > See this post where Rowland suggested using these attributes
to
> > > > store
> > > > the maximum value in LDAP. (These are not automatically
> > > > calculated,
> > > > your script would need to keep them updated.)
> > > >
> > > > - msSFU30MaxUidNumber
> > > > - msSFU30MaxGidNumber
> > >
> > > Just be aware that these are not multi-master update safe to
> > > update.
> > >
> > > If your domain can get out of sync, just be aware that if your
tool
> > > is
> > > pointed at an 'older' DC, it could allocate the same UID
twice.
> >
> > Thanks for the heads up. Am I correct that this is problematic even
> > without the atomic update problem you mention below?
>
> Yes, the atomic update only helps on the single DC.  The joys of multi-
> master replication.
>
> > Let's assume that msSFU30MaxUidNumber is being updated atomically
> > (from the perspective of the LDAP server serving adman's
requests).
> > If
> > I understand your concern correctly, the following sequence could
> > happen:
> >
> > - msSFU30MaxUidNumber = 1000
> > - Adman connects to DC1, assigns UID=1000 to a user, and atomically
> > increments msSFU30MaxUidNumber to 1001
> > - Before the replication completes, Adman connects to DC2, sees the
> > old value of msSFU30MaxUidNumber = 1000, and assigns a duplicate
> > UID=1000 to a different user
> > - msSFU30MaxUidNumber = 1001
>
> Exactly.
>
> > I have a cron job that runs Adman every minute. Since Adman looks up
> > the LDAP server via the SRV record [1] (and doesn't sort them),
it's
> > possible that it could choose a different DC the next time. And I
> > guess if replication was being pokey, this scenario could happen.
> >
> > I'm not sure how to best resolve this. Perhaps one mitigating
method
> > would be to always target the DC with the PDC emulator FSMO role.
> > I've opened issue adman#23 [2] to track this.
>
> I recommend RID based allocation, always take the user RID and add a
> base.  Ideally record that base in the trustPosixOffset attribute.
I agree that RID-based uidNumber/gidNumber assignment probably makes
the most sense. Unfortunately, I have existing domains using the
current counter-based assignment. Perhaps one day I will add RID-based
assignment to ADMan also.
> > > > https://lists.samba.org/archive/samba/2019-June/223499.html
> > > >
> > > > But also, check out my project ADMan, which does several
things,
> > > > one
> > > > of which is assign uidNumber / gidNumber attributes to new
users:
> > > >
> > > > https://gitlab.com/JonathonReinhart/adman/
> > >
> > > Cool.
> > >
> > > One thing I would love is to get the uidNumber/gidNumber
assignment
> > > into Samba.  I would use something like the algorithm from
> > > idmap_sssd
> > > to pick a 'pretty much likely to be unique UID' based on
the RID
> > > and
> > > then store it forever in the uidNumber.
...or maybe I'll add it to Samba instead.
> > One mistake I would love to go back in time to rectify is the
> > suggested UID/GID ranges. I first asked about this here [3]. I'm
> > currently using 100000-200000 for both uidNumber and gidNumber. While
> > this is fine for most use cases, it does preclude the use of
SSSD's
> > "automatic user private groups" feature which, if enabled
> > (auto_private_groups = true), makes it such that "every AD user
has a
> > GID which is identical to the UID." [4] [5] For that reason, I
wish I
> > had gone with disjoint ranges.
>
> Well, if you move to RID based, as users and group share a RID space,
> this becomes available again (for new users).  I agree, keeping UIDs
> and GIDs as a single number space is preferable.
Sigh, does anyone know how to easily mass-update the UID/GID numbers
in Windows ACLs and file ownership on FreeBSD? :-)
> > > In the meantime I would suggest to try to do an LDAP atomic
update
> > > of
> > > the msSFU30MaxUidNumber regardless.
> > > See https://ldapwiki.com/wiki/LDIF%20Atomic%20Operations for what
I
> > > mean.
> >
> > Cool. I had briefly considered the atomicity problems, but kind of
> > ignored them, as Adman is the only agent in my network updating these
> > fields. But this would be a great improvement. I actually identified
> > another related problem while looking at this. I opened adman#24 [6]
> > to track.
> >
I fixed all of the atomicity issues that Andrew pointed out above:

* The "joys of multi-master replication" have been mitigated by
forcing ADMan to only target the DC holding the PDC Emulator FSMO
role. (I run ADMan on my "main" DC with this role, anyway, so this
keeps all of the LDAP traffic local.)

* My LDAP Object Model has been updated to support "atomic"
attributes, which use a "DELETE $old; ADD $new" pattern for updates.
msSFU30MaxUidNumber/msSFU30MaxGidNumber are marked atomic.

* msSFU30MaxUidNumber is now committed before attempting to actually
assign the value to a user/computer.

ADMan v0.8.0 contains these updates:
https://gitlab.com/JonathonReinhart/adman/-/releases/v0.8.0

Thanks again, Andrew.

-- Jonathon