On 2015-11-07 at 17:47 +0000, Jonathan Hunter wrote:> On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote: > > > > Also, for all I know, the DC always has local unix user and group > > IDs, and does NOT use the rfc2307 attributes for this. (Unless > > this has changed recently, but I can't imagine how.) So there is > > nothing wrong with samba not using the rfc ids on the DC -- this is > > how it works by design. > > Thanks Michael. I will see if I can use winbind locally instead of > sssd later this evening, now that I have fully switched to rfc2307 > rather than algorithmic mappings. > > One question on this, though - how is file ownership managed on the DC > from the samba side? I know DCs aren't "supposed" to be used as file > servers in the samba view of things (which is another story > altogether), but I can't understand why sometimes the ID mapping comes > from the rfc2307 attributes and then later on not.I don't understand that yet. As explained in my previous mail, what can happen is that a user first (before given a rfc unixID) gets its uid from the idmap.ldb, but as soon as there is a unixid in the rfc attributes in his ldap object, that should always be used. This is a per-user thing. It is not surpsising that an externally configured sssd (configured to use rfc in a ad, and hence behaving more like a domain member) possibly gives different results, and also consistently uses the rfc attrs, since that one does not have the fallback to idmap.ldb that samba has. The mapping should indeed be consistent for a user on the DC, so it should not intermittently switch between idmap.ldb and the rfc attributes. That would be a bug that we need to understand. One step as written in a previous mail would be to change the dc code to _never_ fall back to idmap.ldb when configured with "idmap_ldb:use rfc2307 = yes". Then you write that when you see the wrong uid, a 'net cache flush' gets you back the correct uid. That is interesting. That flush command flushes winbindd's idmap cache (which is also used r/o by smbd). We need to understand what happens here. Rowland has already correctly asked for higher-level logs. When we have those, we'll see furhter. Cheers - Michael> The mapping needs > to be consistent so that any files on disk are owned by the correct > UID (even if the local DC's Unix system doesn't necessarily know who > that UID is - that's the job of winbindd / sssd / etc. as I understand > it) ?> There are a lot of people (including me) who for various reasons > really, really want to use a single machine as both a DC and a file > server. Having this work with any sort of consistency in UID mappings > is proving to be a little bit problematic :) > > It's frustrating for me because it works for a while (5 months until > yesterday) but then something triggers and it doesn't work again...-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151107/eac824e5/signature.sig>
On 7 November 2015 at 18:57, Michael Adam <obnox at samba.org> wrote:> The mapping should indeed be consistent for a user on the DC, > so it should not intermittently switch between idmap.ldb and > the rfc attributes. That would be a bug that we need to > understand. > > One step as written in a previous mail would be to change > the dc code to _never_ fall back to idmap.ldb when configured > with "idmap_ldb:use rfc2307 = yes". > > Then you write that when you see the wrong uid, > a 'net cache flush' gets you back the correct uid. > That is interesting. That flush command flushes > winbindd's idmap cache (which is also used r/o by > smbd). > > We need to understand what happens here. > Rowland has already correctly asked for higher-level > logs.I now have these logs available - I will email the full extracts direct to you Michael, but I don't want to post them on a public forum as there is rather a lot of information to sanitise and I will no doubt break consistency in the logs if I do so. If anybody else on the dev team would like a copy, that's fine - just email me and I will send them to you also. In summary, /usr/local/samba/var/log.winbindd-idmap shows the following 3000007 UID being allocated to my user that already has a UID from rfc2307 attributes e.g. 41234. I don't know why.. but this is where the problem occurs: [2015/11/08 01:07:02.077532, 4, pid=24816, effective(0, 0), real(0, 0), class=winbind] ../source3/winbindd/winbindd_dual.c:1389(child_handler) child daemon request 59 [2015/11/08 01:07:02.077573, 10, pid=24816, effective(0, 0), real(0, 0), class=winbind] ../source3/winbindd/winbindd_dual.c:512(child_process_request) child_process_request: request fn NDRCMD [2015/11/08 01:07:02.077616, 10, pid=24816, effective(0, 0), real(0, 0), class=winbind] ../source3/winbindd/winbindd_dual_ndr.c:315(winbindd_dual_ndrcmd) winbindd_dual_ndrcmd: Running command WBINT_GID2SID (no domain) [2015/11/08 01:07:02.077666, 10, pid=24816, effective(0, 0), real(0, 0), class=idmap] ../source3/winbindd/idmap_util.c:106(idmap_gid_to_sid) idmap_gid_to_sid: gid = [3000007] [2015/11/08 01:07:02.077732, 10, pid=24816, effective(0, 0), real(0, 0), class=idmap] ../source3/winbindd/idmap.c:518(idmap_backends_unixid_to_sid) idmap_backend_unixid_to_sid: xid = 3000007 (type 2) [2015/11/08 01:07:02.081098, 4, pid=24816, effective(0, 0), real(0, 0), class=winbind] ../source3/winbindd/winbindd_dual.c:1397(child_handler) Finished processing child request 59 I can raise this in bugzilla if that is helpful, now. My working theory is that there is maybe a file on the server, somewhere, that is mapped to an old UID i.e. 3000007, either via an acl or by simple UID ownership.. and in some way samba is coming across this file and it's triggering the behaviour above?? I think the previous suggestion of never falling back to idmap.ldb if rfc2307 is configured makes perfect sense. Perhaps a warning should be printed if this kind of situation is encountered (assuming we can detect it!) Thanks! Jonathan -- "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein
On 2015-11-08 at 03:13 +0000, Jonathan Hunter wrote:> On 7 November 2015 at 18:57, Michael Adam <obnox at samba.org> wrote: > > The mapping should indeed be consistent for a user on the DC, > > so it should not intermittently switch between idmap.ldb and > > the rfc attributes. That would be a bug that we need to > > understand. > > > > One step as written in a previous mail would be to change > > the dc code to _never_ fall back to idmap.ldb when configured > > with "idmap_ldb:use rfc2307 = yes". > > > > Then you write that when you see the wrong uid, > > a 'net cache flush' gets you back the correct uid. > > That is interesting. That flush command flushes > > winbindd's idmap cache (which is also used r/o by > > smbd). > > > > We need to understand what happens here. > > Rowland has already correctly asked for higher-level > > logs. > > I now have these logs available - I will email the full extracts > direct to you Michael, but I don't want to post them on a public forum > as there is rather a lot of information to sanitise and I will no > doubt break consistency in the logs if I do so. If anybody else on the > dev team would like a copy, that's fine - just email me and I will > send them to you also. > > In summary, /usr/local/samba/var/log.winbindd-idmap shows the > following 3000007 UID being allocated to my user that already has a > UID from rfc2307 attributes e.g. 41234. I don't know why.. but this is > where the problem occurs: > > [2015/11/08 01:07:02.077532, 4, pid=24816, effective(0, 0), real(0, > 0), class=winbind] > ../source3/winbindd/winbindd_dual.c:1389(child_handler) > child daemon request 59 > [2015/11/08 01:07:02.077573, 10, pid=24816, effective(0, 0), real(0, > 0), class=winbind] > ../source3/winbindd/winbindd_dual.c:512(child_process_request) > child_process_request: request fn NDRCMD > [2015/11/08 01:07:02.077616, 10, pid=24816, effective(0, 0), real(0, > 0), class=winbind] > ../source3/winbindd/winbindd_dual_ndr.c:315(winbindd_dual_ndrcmd) > winbindd_dual_ndrcmd: Running command WBINT_GID2SID (no domain) > [2015/11/08 01:07:02.077666, 10, pid=24816, effective(0, 0), real(0, > 0), class=idmap] > ../source3/winbindd/idmap_util.c:106(idmap_gid_to_sid) > idmap_gid_to_sid: gid = [3000007] > [2015/11/08 01:07:02.077732, 10, pid=24816, effective(0, 0), real(0, > 0), class=idmap] > ../source3/winbindd/idmap.c:518(idmap_backends_unixid_to_sid) > idmap_backend_unixid_to_sid: xid = 3000007 (type 2) > [2015/11/08 01:07:02.081098, 4, pid=24816, effective(0, 0), real(0, > 0), class=winbind] > ../source3/winbindd/winbindd_dual.c:1397(child_handler) > Finished processing child request 59 > > I can raise this in bugzilla if that is helpful, now. > > My working theory is that there is maybe a file on the server, > somewhere, that is mapped to an old UID i.e. 3000007, either via an > acl or by simple UID ownership.. and in some way samba is coming > across this file and it's triggering the behaviour above??It looks like a gid_to_sid call triggers the problem. Interestingly, gid not uid. And this seems to be coming from nsswitch, may e.g. have been a ls -l on a file with 3000007 as GID. That makes perfect sense, with this possible chain of events: - user created. no rfc attribute configured yet. - user accesses dc, mapping sid<->3000007 is created. - the mappings of idmap.ldb are can serve for both uids and gids. hence the gid can have ended up in the group perms of a file somehow. - now the rfc uid is set for the user. and later access as the user also fills the idmap cache with sid<->40000X. - for the time being this user is always treated as uid 40000X. - but then somehting like ls -l triggers the getgrgid 3000007 call and this leads to gid_to_sid(3000007) in winbindd. Since this is coming via Unix IDs it finds the entry 3000007 in idmap.ldb and returns that mapping, also filling the cache with that mapping. - henceforth (until the cache expires) idmap requests for the SID are answered from the cache and 3000007 is returned instead of the 400000X. You could try to prove this by: - net cache flush - getent passwd <your user> - net cache list | grep <your.sid> - ls -l on the file with gid 3000007 - net cache list | grep <your.sid> A more 'realistic' variant would be: - find a file with gid 3000007 in your tree. - net cache flush - access the samba with your user - net cache list | grep <your.sid> - getent group 3000007 - net cache list | grep <your.sid> Also dump of idmap entries from idmap.ldb with id 3000007 could be interesting.> I think the previous suggestion of never falling back to idmap.ldb if > rfc2307 is configured makes perfect sense.Indeed. That would have prevented the above problem! This is also the reason why in the s3-winbindd code we do not use these layered idmap systems. (This here is the s4-winbind idmap code.) So as part of solving the problem, I think we need to disable the fallback to idmap.ldb. The above chain of events could be turned into a nice general reproducer and that might be sufficient to create a bug and justify the patch. I'd love to hear the experts of the AD code comment on it though. Cheers - Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151108/5d881d63/signature.sig>
On 07/11/15 19:57, Michael Adam wrote:> On 2015-11-07 at 17:47 +0000, Jonathan Hunter wrote: >> On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote: >>> Also, for all I know, the DC always has local unix user and group >>> IDs, and does NOT use the rfc2307 attributes for this. (Unless >>> this has changed recently, but I can't imagine how.) So there is >>> nothing wrong with samba not using the rfc ids on the DC -- this is >>> how it works by design. >> Thanks Michael. I will see if I can use winbind locally instead of >> sssd later this evening, now that I have fully switched to rfc2307 >> rather than algorithmic mappings. >> >> One question on this, though - how is file ownership managed on the DC >> from the samba side? I know DCs aren't "supposed" to be used as file >> servers in the samba view of things (which is another story >> altogether), but I can't understand why sometimes the ID mapping comes >> from the rfc2307 attributes and then later on not. > I don't understand that yet. > As explained in my previous mail, what can happen is that > a user first (before given a rfc unixID) gets its uid from the > idmap.ldb, but as soon as there is a unixid in the rfc > attributes in his ldap object, that should always be used. > > This is a per-user thing. > > It is not surpsising that an externally configured sssd > (configured to use rfc in a ad, and hence behaving more like > a domain member) possibly gives different results, and also > consistently uses the rfc attrs, since that one does not > have the fallback to idmap.ldb that samba has.sssd's uses its own implementation of winbind and _always_ retrieves the same id from AD. Repeat, _always_. Currently it and nslcd are the only way to obtain full rfc2307 and consistent ids on DCs. Neither winbind nor winbindd can do so.> > The mapping should indeed be consistent for a user on the DC, > so it should not intermittently switch between idmap.ldb and > the rfc attributes. That would be a bug that we need to > understand. > > One step as written in a previous mail would be to change > the dc code to _never_ fall back to idmap.ldb when configured > with "idmap_ldb:use rfc2307 = yes".It has to fall back as uid is undefined for builtin objects.
On 2015-11-08 at 20:34 +0100, buhorojo wrote:> On 07/11/15 19:57, Michael Adam wrote: > >On 2015-11-07 at 17:47 +0000, Jonathan Hunter wrote: > >>On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote: > >>>Also, for all I know, the DC always has local unix user and group > >>>IDs, and does NOT use the rfc2307 attributes for this. (Unless > >>>this has changed recently, but I can't imagine how.) So there is > >>>nothing wrong with samba not using the rfc ids on the DC -- this is > >>>how it works by design. > >>Thanks Michael. I will see if I can use winbind locally instead of > >>sssd later this evening, now that I have fully switched to rfc2307 > >>rather than algorithmic mappings. > >> > >>One question on this, though - how is file ownership managed on the DC > >>from the samba side? I know DCs aren't "supposed" to be used as file > >>servers in the samba view of things (which is another story > >>altogether), but I can't understand why sometimes the ID mapping comes > >>from the rfc2307 attributes and then later on not. > >I don't understand that yet. > >As explained in my previous mail, what can happen is that > >a user first (before given a rfc unixID) gets its uid from the > >idmap.ldb, but as soon as there is a unixid in the rfc > >attributes in his ldap object, that should always be used. > > > >This is a per-user thing. > > > >It is not surpsising that an externally configured sssd > >(configured to use rfc in a ad, and hence behaving more like > >a domain member) possibly gives different results, and also > >consistently uses the rfc attrs, since that one does not > >have the fallback to idmap.ldb that samba has. > sssd's uses its own implementation of winbindI repeat: sssd does not implement winbind. It implements some parts of the winbind protocol. It is not a drop-in replacement for winbind(d). And the ad-dc forcefully uses winbindd anyways, so sssd is not at all an option.> and _always_ retrieves the same id from AD. Repeat, _always_. > Currently it and nslcd are the only way to obtain full rfc2307 > and consistent ids on DCs. Neither winbind nor > winbindd can do so.Sure. winbindd can do it. use the idmap_ad module :-) The only problem exists in the ad/dc setup due to the fallback (that I proposed to remove).> >The mapping should indeed be consistent for a user on the DC, > >so it should not intermittently switch between idmap.ldb and > >the rfc attributes. That would be a bug that we need to > >understand. > > > >One step as written in a previous mail would be to change > >the dc code to _never_ fall back to idmap.ldb when configured > >with "idmap_ldb:use rfc2307 = yes". > It has to fall back as uid is undefined for builtin objects.As someone else has already said: builtin are different. These are machine-local anyways. We were talking about the main user DB, subordinate to the domain SID. And for _that_, it should not fall back. Fallback is a bug here as elaborated ealier... Cheers - Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151108/7af67aa3/signature.sig>