On 2015-11-07 at 12:37 +0000, Rowland Penny wrote:> On 07/11/15 11:31, Jonathan Hunter wrote: > >On 7 November 2015 at 10:11, Rowland Penny <rowlandpenny241155 at gmail.com> wrote: > >>Is it possible that sssd is failing? > >>What do you have in /etc/nsswitch? > ># cat /etc/nsswitch.conf | egrep "(passwd|group)" > >passwd: files sss > >group: files sss > > > >But I don't think this is anything to do with sssd. As I understand it: > > > >Local machine UNIX use (i.e. logging in via ssh; looking at files on > >disk via "ls"; etc.) uses sssd, because this is what I have set in > >nsswitch.conf. This all works fine, I have no problems with this. > > > >"SMB file access" (i.e. a Windows client machine elsewhere on the > >network, accessing resources via \\server\share\path) does not use > >sssd, but uses smbd + winbind/winbindd for UID resolution? This is the > >part that is failing intermittently. > > > >>It could be that sssd isn't running or running correctly, so it cannot get > >>the required info from AD, so winbind is returning the info from idmap.ldb, > >>hence the '3000000' numbers. > >Does winbind/wbinfo ever query what is defined in /etc/nsswitch.conf, > >or does it always use the samba internal UID resolution? I thought it > >would bypass nsswitch.conf entirely - hence my suspicion that this is > >nothing to do with sssd. > > > >It's hard to reproduce this at will - right now "wbinfo -i myuser" is > >returning correct UID information. The problem (as far as i can tell) > >is that, every so often, despite me having "idmap_ldb:use rfc2307 > >yes" in smb.conf, this same wbinfo command returns incorrect UID > >information (as also shown in "net cache list") and therefore this is > >why I cannot access files via smbd until I clear the idmap cache via > >"net cache flush". > > > >I'm trying to narrow it down to a particular set of circumstances but > >it's so intermittent, I'm really struggling. > > > >I would raise a bug on bugzilla but I'm not sure there's enough > >information here for someone familiar with the code to resolve it, > >yet. > > > >It is of course possible that I'm doing something wrong - but the > >thing that makes me convinced it's a bug is that I have /not/ changed > >my configuration in any way since June (when I last saw this issue). > >After my recent upgrade to 4.3 the problem came back - I saw it again > >last night - but has not reoccurred since then until now.. I really do > >think there is a subtle bug here. > > > >Is it worth me putting all this into a bugzilla entry, even though I > >haven't yet narrowed down the full circumstances under which it > >happens? > > > >Thanks > > > >Jonathan > > > > The problem is, sssd now uses its own version of winbind, this came in (I > believe) with version 1.12.0 but I 'think' red-hat backport some things to > earlier versions. As I understand it, you will be probably be using > 1.11.6-30 and it is the '30' that says what it contains, perhaps you are > using winbindd and don't realise it.As of Version 4.2, the Samba AD/DC is using winbindd by default. It is started by the samba daemon. Samba is designed to work with winbindd. sssd does not contain its own winbind, but it implements some parts of the winbind protocol. So my suggestion is to remove sss from /etc/nsswitch.conf and use winbind instead. This is how it is designed to work. Also, for all I know, the DC always has local unix user and group IDs, and does NOT use the rfc2307 attributes for this. (Unless this has changed recently, but I can't imagine how.) So there is nothing wrong with samba not using the rfc ids on the DC -- this is how it works by design. These rfc uids/gids can be used on the member servers! Cheers - Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151107/1f1f469f/signature.sig>
On 07/11/15 17:01, Michael Adam wrote:> On 2015-11-07 at 12:37 +0000, Rowland Penny wrote: >> On 07/11/15 11:31, Jonathan Hunter wrote: >>> On 7 November 2015 at 10:11, Rowland Penny <rowlandpenny241155 at gmail.com> wrote: >>>> Is it possible that sssd is failing? >>>> What do you have in /etc/nsswitch? >>> # cat /etc/nsswitch.conf | egrep "(passwd|group)" >>> passwd: files sss >>> group: files sss >>> >>> But I don't think this is anything to do with sssd. As I understand it: >>> >>> Local machine UNIX use (i.e. logging in via ssh; looking at files on >>> disk via "ls"; etc.) uses sssd, because this is what I have set in >>> nsswitch.conf. This all works fine, I have no problems with this. >>> >>> "SMB file access" (i.e. a Windows client machine elsewhere on the >>> network, accessing resources via \\server\share\path) does not use >>> sssd, but uses smbd + winbind/winbindd for UID resolution? This is the >>> part that is failing intermittently. >>> >>>> It could be that sssd isn't running or running correctly, so it cannot get >>>> the required info from AD, so winbind is returning the info from idmap.ldb, >>>> hence the '3000000' numbers. >>> Does winbind/wbinfo ever query what is defined in /etc/nsswitch.conf, >>> or does it always use the samba internal UID resolution? I thought it >>> would bypass nsswitch.conf entirely - hence my suspicion that this is >>> nothing to do with sssd. >>> >>> It's hard to reproduce this at will - right now "wbinfo -i myuser" is >>> returning correct UID information. The problem (as far as i can tell) >>> is that, every so often, despite me having "idmap_ldb:use rfc2307 >>> yes" in smb.conf, this same wbinfo command returns incorrect UID >>> information (as also shown in "net cache list") and therefore this is >>> why I cannot access files via smbd until I clear the idmap cache via >>> "net cache flush". >>> >>> I'm trying to narrow it down to a particular set of circumstances but >>> it's so intermittent, I'm really struggling. >>> >>> I would raise a bug on bugzilla but I'm not sure there's enough >>> information here for someone familiar with the code to resolve it, >>> yet. >>> >>> It is of course possible that I'm doing something wrong - but the >>> thing that makes me convinced it's a bug is that I have /not/ changed >>> my configuration in any way since June (when I last saw this issue). >>> After my recent upgrade to 4.3 the problem came back - I saw it again >>> last night - but has not reoccurred since then until now.. I really do >>> think there is a subtle bug here. >>> >>> Is it worth me putting all this into a bugzilla entry, even though I >>> haven't yet narrowed down the full circumstances under which it >>> happens? >>> >>> Thanks >>> >>> Jonathan >>> >> The problem is, sssd now uses its own version of winbind, this came in (I >> believe) with version 1.12.0 but I 'think' red-hat backport some things to >> earlier versions. As I understand it, you will be probably be using >> 1.11.6-30 and it is the '30' that says what it contains, perhaps you are >> using winbindd and don't realise it. > As of Version 4.2, the Samba AD/DC is using winbindd by default. > It is started by the samba daemon. Samba is designed to work > with winbindd. sssd does not contain its own winbind, but it > implements some parts of the winbind protocol. So my suggestion > is to remove sss from /etc/nsswitch.conf and use winbind instead. > This is how it is designed to work. > > Also, for all I know, the DC always has local unix user and group > IDs, and does NOT use the rfc2307 attributes for this. (Unless > this has changed recently, but I can't imagine how.) So there is > nothing wrong with samba not using the rfc ids on the DC -- this is > how it works by design. >On the DC the users & groups get xidNumbers from idmap.ldb, but if they are given uidNumbers and gidNumbers, these will be used instead. As far as I am aware, it has always worked like this. The only problem with using the DC as a fileserver is that the users unixhomedirectory and loginshell are ignored, this is why people use sssd etc, if somebody (and don't ask me, to me 'C' comes between 'B' & 'D') would make samba on the DC use all the RFC2307 attributes, there would be no need to use sssd or nlscd. Rowland
On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote:> > Also, for all I know, the DC always has local unix user and group > IDs, and does NOT use the rfc2307 attributes for this. (Unless > this has changed recently, but I can't imagine how.) So there is > nothing wrong with samba not using the rfc ids on the DC -- this is > how it works by design.Thanks Michael. I will see if I can use winbind locally instead of sssd later this evening, now that I have fully switched to rfc2307 rather than algorithmic mappings. One question on this, though - how is file ownership managed on the DC from the samba side? I know DCs aren't "supposed" to be used as file servers in the samba view of things (which is another story altogether), but I can't understand why sometimes the ID mapping comes from the rfc2307 attributes and then later on not. The mapping needs to be consistent so that any files on disk are owned by the correct UID (even if the local DC's Unix system doesn't necessarily know who that UID is - that's the job of winbindd / sssd / etc. as I understand it) ? There are a lot of people (including me) who for various reasons really, really want to use a single machine as both a DC and a file server. Having this work with any sort of consistency in UID mappings is proving to be a little bit problematic :) It's frustrating for me because it works for a while (5 months until yesterday) but then something triggers and it doesn't work again... Cheers J -- "If we knew what it was we were doing, it would not be called research, would it?" - Albert Einstein
On 07/11/15 17:47, Jonathan Hunter wrote:> On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote: >> Also, for all I know, the DC always has local unix user and group >> IDs, and does NOT use the rfc2307 attributes for this. (Unless >> this has changed recently, but I can't imagine how.) So there is >> nothing wrong with samba not using the rfc ids on the DC -- this is >> how it works by design. > Thanks Michael. I will see if I can use winbind locally instead of > sssd later this evening, now that I have fully switched to rfc2307 > rather than algorithmic mappings. > > One question on this, though - how is file ownership managed on the DC > from the samba side? I know DCs aren't "supposed" to be used as file > servers in the samba view of things (which is another story > altogether), but I can't understand why sometimes the ID mapping comes > from the rfc2307 attributes and then later on not. The mapping needs > to be consistent so that any files on disk are owned by the correct > UID (even if the local DC's Unix system doesn't necessarily know who > that UID is - that's the job of winbindd / sssd / etc. as I understand > it) ? > > There are a lot of people (including me) who for various reasons > really, really want to use a single machine as both a DC and a file > server. Having this work with any sort of consistency in UID mappings > is proving to be a little bit problematic :) > > It's frustrating for me because it works for a while (5 months until > yesterday) but then something triggers and it doesn't work again... > > Cheers > > J >The problem here is that whilst the uidNumbers & gidNumbers have always been consistent when used on a DC with winbind (now winbindd), you have never been able to use per-user home dirs and login shells. The user ID problem on DCs using xidNumbers from idmap.ldb is compounded by the fact that idmap.ldb can be and usually is different on DCs. The only way to get consistent IDs is to use RFC2307 attributes, but as I said, you cannot use the unixhomedirectory and loginshell attributes on a DC. Rowland
On 2015-11-07 at 17:11 +0000, Rowland Penny wrote:> On 07/11/15 17:01, Michael Adam wrote: > >On 2015-11-07 at 12:37 +0000, Rowland Penny wrote: > >>On 07/11/15 11:31, Jonathan Hunter wrote: > >>>On 7 November 2015 at 10:11, Rowland Penny <rowlandpenny241155 at gmail.com> wrote: > >>>>Is it possible that sssd is failing? > >>>>What do you have in /etc/nsswitch? > >>># cat /etc/nsswitch.conf | egrep "(passwd|group)" > >>>passwd: files sss > >>>group: files sss > >>> > >>>But I don't think this is anything to do with sssd. As I understand it: > >>> > >>>Local machine UNIX use (i.e. logging in via ssh; looking at files on > >>>disk via "ls"; etc.) uses sssd, because this is what I have set in > >>>nsswitch.conf. This all works fine, I have no problems with this. > >>> > >>>"SMB file access" (i.e. a Windows client machine elsewhere on the > >>>network, accessing resources via \\server\share\path) does not use > >>>sssd, but uses smbd + winbind/winbindd for UID resolution? This is the > >>>part that is failing intermittently. > >>> > >>>>It could be that sssd isn't running or running correctly, so it cannot get > >>>>the required info from AD, so winbind is returning the info from idmap.ldb, > >>>>hence the '3000000' numbers. > >>>Does winbind/wbinfo ever query what is defined in /etc/nsswitch.conf, > >>>or does it always use the samba internal UID resolution? I thought it > >>>would bypass nsswitch.conf entirely - hence my suspicion that this is > >>>nothing to do with sssd. > >>> > >>>It's hard to reproduce this at will - right now "wbinfo -i myuser" is > >>>returning correct UID information. The problem (as far as i can tell) > >>>is that, every so often, despite me having "idmap_ldb:use rfc2307 > >>>yes" in smb.conf, this same wbinfo command returns incorrect UID > >>>information (as also shown in "net cache list") and therefore this is > >>>why I cannot access files via smbd until I clear the idmap cache via > >>>"net cache flush". > >>> > >>>I'm trying to narrow it down to a particular set of circumstances but > >>>it's so intermittent, I'm really struggling. > >>> > >>>I would raise a bug on bugzilla but I'm not sure there's enough > >>>information here for someone familiar with the code to resolve it, > >>>yet. > >>> > >>>It is of course possible that I'm doing something wrong - but the > >>>thing that makes me convinced it's a bug is that I have /not/ changed > >>>my configuration in any way since June (when I last saw this issue). > >>>After my recent upgrade to 4.3 the problem came back - I saw it again > >>>last night - but has not reoccurred since then until now.. I really do > >>>think there is a subtle bug here. > >>> > >>>Is it worth me putting all this into a bugzilla entry, even though I > >>>haven't yet narrowed down the full circumstances under which it > >>>happens? > >>> > >>>Thanks > >>> > >>>Jonathan > >>> > >>The problem is, sssd now uses its own version of winbind, this came in (I > >>believe) with version 1.12.0 but I 'think' red-hat backport some things to > >>earlier versions. As I understand it, you will be probably be using > >>1.11.6-30 and it is the '30' that says what it contains, perhaps you are > >>using winbindd and don't realise it. > >As of Version 4.2, the Samba AD/DC is using winbindd by default. > >It is started by the samba daemon. Samba is designed to work > >with winbindd. sssd does not contain its own winbind, but it > >implements some parts of the winbind protocol. So my suggestion > >is to remove sss from /etc/nsswitch.conf and use winbind instead. > >This is how it is designed to work. > > > >Also, for all I know, the DC always has local unix user and group > >IDs, and does NOT use the rfc2307 attributes for this. (Unless > >this has changed recently, but I can't imagine how.) So there is > >nothing wrong with samba not using the rfc ids on the DC -- this is > >how it works by design. > > > > On the DC the users & groups get xidNumbers from idmap.ldb, but if they are > given uidNumbers and gidNumbers, these will be used instead. As far as I am > aware, it has always worked like this.Possibly. The current sid_to_unixid code works like this: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ if config has "idmap_ldb:use rfc2307 = true": if sam.ldb contains rfc uid/gid in AD object: use it # fall through if idmap.ldb contains mapping use it else: create new mapping in idmap.ldb use it ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ So there is a config option, which default to "false" but is set to true in provision if --use-rfc2307 was given to the provision cmdline. But if this is enabled, this is really bad, especially when using the DC extensively as a file server (which should avoided anyways, if possible): - Assume a user is created in AD. - That user is not immediately given a rfc user id. - The users accesses the DC as file server. A idmap.ldb entrie is created and these ids end up in the files the user creates. - Later the admin gives this user an rfc user id. (different from the idmap.ldb uid) - Next time the user accesses the DC as file server, it can not access his previously created files. I may have missed somthing in the code, but it looks like this to me. So I would currently suggest to avoid: a) using the DC as a file server (I suggest that anyways) b) using the rfc attributes on the DC itself.> The only problem with using the DC as a fileserver is that the users > unixhomedirectory and loginshell are ignored,Not the only. See above.> this is why people use sssd > etc, if somebody (and don't ask me, to me 'C' comes between 'B' & 'D') would > make samba on the DC use all the RFC2307 attributes, there would be no need > to use sssd or nlscd.It should not be too big a problem to use the homedir etc attribs in principle. I think we could and (should) do that. At least controlled by a config option (possibly the same as above). But the biggest problem I see is the creation of the unixIDs: I do not see how this can be reliably done, at least in a multi-DC environment. And for sure not in a heterogenious one (with Samba and Windows DCs): Assume that we change the DC to only use the rfc UIDs so that the fallback to idmap.ldb described above is not done. (I think this is the only reasonable way to go about it.) Then a newly created account is useless until it is given a unix ID in AD. So my assumption is that this should be done automatically when the user / group is created. Now the problem with multiple DCs is that this is a local operation changing the global database which is later replicated with other DCs. There is no mechanism (I am aware of) in AD to protect from two admins on two different dcs to set the same UID on different users. For RIDs there is the rid master FSMO role, and a similar concept would be needed for unixids in order to make this reliable. But there is no such thing in AD. For a pure Samba domain, we could probably invent our own protocol extension for doing this, but it would not be possible to use it in a heterogenious environment (which may not be too bad), but it is also not trivial to implement. Because of these fundamental problems, I repeat my pov: - don't use the rfc attrs on the dc (at least not by default) - don't use the DC as a file server In a small domain, with a single admin who conciously creates all users and groups, ideally always on the same DC, all these problems can be avoided. But when we proved such mechanisms in Samba, the should be rock solid, also in a big multi-DC multi-Admin environment. So I think what we could do is this: - make "idmap_ldb:use rfc2307" strict (no fallback to idmap.ldb) - make this parameter (or a new one) use the other unix attrs from ad This would then still need manual creation of the unix IDs, but it would be a first step. Cheers - Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151107/289c3ab4/signature.sig>
On 2015-11-07 at 17:47 +0000, Jonathan Hunter wrote:> On 7 November 2015 at 17:01, Michael Adam <obnox at samba.org> wrote: > > > > Also, for all I know, the DC always has local unix user and group > > IDs, and does NOT use the rfc2307 attributes for this. (Unless > > this has changed recently, but I can't imagine how.) So there is > > nothing wrong with samba not using the rfc ids on the DC -- this is > > how it works by design. > > Thanks Michael. I will see if I can use winbind locally instead of > sssd later this evening, now that I have fully switched to rfc2307 > rather than algorithmic mappings. > > One question on this, though - how is file ownership managed on the DC > from the samba side? I know DCs aren't "supposed" to be used as file > servers in the samba view of things (which is another story > altogether), but I can't understand why sometimes the ID mapping comes > from the rfc2307 attributes and then later on not.I don't understand that yet. As explained in my previous mail, what can happen is that a user first (before given a rfc unixID) gets its uid from the idmap.ldb, but as soon as there is a unixid in the rfc attributes in his ldap object, that should always be used. This is a per-user thing. It is not surpsising that an externally configured sssd (configured to use rfc in a ad, and hence behaving more like a domain member) possibly gives different results, and also consistently uses the rfc attrs, since that one does not have the fallback to idmap.ldb that samba has. The mapping should indeed be consistent for a user on the DC, so it should not intermittently switch between idmap.ldb and the rfc attributes. That would be a bug that we need to understand. One step as written in a previous mail would be to change the dc code to _never_ fall back to idmap.ldb when configured with "idmap_ldb:use rfc2307 = yes". Then you write that when you see the wrong uid, a 'net cache flush' gets you back the correct uid. That is interesting. That flush command flushes winbindd's idmap cache (which is also used r/o by smbd). We need to understand what happens here. Rowland has already correctly asked for higher-level logs. When we have those, we'll see furhter. Cheers - Michael> The mapping needs > to be consistent so that any files on disk are owned by the correct > UID (even if the local DC's Unix system doesn't necessarily know who > that UID is - that's the job of winbindd / sssd / etc. as I understand > it) ?> There are a lot of people (including me) who for various reasons > really, really want to use a single machine as both a DC and a file > server. Having this work with any sort of consistency in UID mappings > is proving to be a little bit problematic :) > > It's frustrating for me because it works for a while (5 months until > yesterday) but then something triggers and it doesn't work again...-------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: <http://lists.samba.org/pipermail/samba/attachments/20151107/eac824e5/signature.sig>