Sven Schwedas
2019-May-21 09:44 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
Once again, something with Samba thirty bazillion components broke. Once again, my choices for logging are "nothing" or "15 MB/s spread of ten different files, because 'client authentication failed' totally needs to be lower priority than malloc debug info". Once again, none of these messages is actually able to convey what broke, where, why. Why is it impossible for Samba to provide useful error reporting? How is anyone supposed to use this in production? Why is the only way we have to troubleshoot is having Rowland and LPH bike shedding over smb.conf formatting styles for two weeks until the problem randomly solves itself? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20190521/088033ac/signature.sig>
Rowland penny
2019-May-21 10:12 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
On 21/05/2019 10:44, Sven Schwedas via samba wrote:> Once again, something with Samba thirty bazillion components broke. Once > again, my choices for logging are "nothing" or "15 MB/s spread of ten > different files, because 'client authentication failed' totally needs to > be lower priority than malloc debug info". Once again, none of these > messages is actually able to convey what broke, where, why. Why is it > impossible for Samba to provide useful error reporting? How is anyone > supposed to use this in production? Why is the only way we have to > troubleshoot is having Rowland and LPH bike shedding over smb.conf > formatting styles for two weeks until the problem randomly solves itself?Hi, its the ace bike shedder here ;-) The main reason why we have to 'bike shed' is the lack of info provided. Many initial posts basically start with 'help, it doesn't work', no mention of what doesn't work or why, no mention of OS or version of Samba and sometimes getting information can feel like extracting teeth ;-) You can get better logging, can I suggest you read the manpage for smb.conf: man smb.conf Pay attention to these parameters: log file, logging, log level, max log size, syslog only, syslog Having said all that, what OS and Samba version? What is in your smb.conf and most importantly, what isn't working with authentication ? Rowland
L.P.H. van Belle
2019-May-21 10:20 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
Hai Sven, I do agree on somepoint here, but not all. First off all, samba does produce usefull reporting info. Depending on log levels you have set. ( man smb.conf /log level ). And then again we always have to ask, can you show the smb.conf. Not that i dont believe you have wrong setting but because i must see you have good setttings. And the problem is not always/only samba, keep in mind that the current windows updates are not as stable as years ago. My personal opinion here, current windows updates are really bad in quality, but why is this... This is because MS is testing on users computers, that simple. You dont believe this.. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PolicyManager\current\device\System The key used: (AllowExperimentation) About, client authentication failed... Windows 10, and are use useing SMB1 then disable the autoremove of smb1 in windows10. Dont forget, i use windows since windows 2.1, i've seen enough with windows. The current changes pointing at NTLM V1/V2 a lot has been done by MS and Samba. Simple things changed.. For example https://support.microsoft.com/nl-nl/help/3181029/smb-file-server-share-access-is-unsuccessful-through-dns-cname-alias I would preffer also to see a clear difference in settings in manuals for samba-AD-DC samba-AD-Member-file server samba-AD-Member-Print server samba-AD-Member-authonly server Samba-NT4-PDC Samba-NT4-BDC Samba-NT4-Member Samba-Standalone Now, add to this the problem Rowland and i are looking into, found recently. A "renamed" computer, might have or is missing or has wrong settings in AD. A common problem we found yesterday, that also exists in windows itself and caused by windows clients. And these found bugs found in windows are inherreted in samba, not always.. But in this case it is. Sorry if you feel bad about this but we are here to help, in such cases. So if you have ideas to improve the debugging. This is on github. https://github.com/thctlo/samba4/blob/master/samba-collect-debug-info.sh Fork/add you parts. And yes, the script really helps in quicker finding the problems. You can ask that here at list users.. So far.. Time to lunch.. Greetz, Louis> -----Oorspronkelijk bericht----- > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Sven > Schwedas via samba > Verzonden: dinsdag 21 mei 2019 11:44 > Aan: samba at lists.samba.org > Onderwerp: [Samba] Debugging Samba is a total PITA and this > needs to improve > > Once again, something with Samba thirty bazillion components broke. Once > again, my choices for logging are "nothing" or "15 MB/s spread of ten > different files, because 'client authentication failed' totally needs to > be lower priority than malloc debug info". Once again, none of these > messages is actually able to convey what broke, where, why. Why is it > impossible for Samba to provide useful error reporting? How is anyone > supposed to use this in production? Why is the only way we have to > troubleshoot is having Rowland and LPH bike shedding over smb.conf > formatting styles for two weeks until the problem randomly > solves itself? > > -- > To unsubscribe from this list go to the following URL and read the > instructions: https://lists.samba.org/mailman/options/samba >
Rowland penny
2019-May-21 10:47 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
On 21/05/2019 11:20, L.P.H. van Belle via samba wrote:> Now, add to this the problem Rowland and i are looking into, found recently. > A "renamed" computer, might have or is missing or has wrong settings in AD. > A common problem we found yesterday, that also exists in windows itself and caused by windows clients. > And these found bugs found in windows are inherreted in samba, not always.. But in this case it is.To add to this, if you rename a domain computer on Windows, you do not get the following changed in the Samba AD object: dn, cn, name and distinguishedname Whilst if you use ldbrename on an AD DC, exactly the opposite happens and you cannot tell the difference by examining the computer object. The main problem I have with this is, is this a windows problem or a Samba problem ? I cannot find anything that definitely says that when you rename a computer, everything should be changed. Lots of webpages that infer this is the case, but nothing that shows a computer object before and after a rename, or lists what should be changed. Rowland
Sven Schwedas
2019-May-21 11:27 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
The smb.conf hasn't changed since the last three or four times I've posted here asking for help: https://up.tao.at/u/samba/villach-file.txt Top level error I'm seeing is that since today *some* Windows users are denied SMB access to this one member server ("Network password is invalid"), but not all users. Worked fine before today. wbinfo -p/-P work, wbinfo -a shows the same problem of some users working, some not: Those that do work, report success with plaintext auth, and NT_STATUS_WRONG_PASSWORD for challenge/response auth (wtf?). Those that don't work at all, fail plaintext auth and report NT_STATUS_INTERNAL_DB_CORRUPTION for challenge/response. Not sure if that means anything, given that challenge/response seems to always fail with nonsensical error messages. All the other working member servers also report NT_STATUS_WRONG_PASSWORD for c/r auth. 15 MB/s error logs were not an exaggeration, BTW, that's what I saw when I cranked up the logging level to 10, since the default log level didn't bother even reporting the logon failures at all (which should be sensible defaults, but oh well). Since I don't know what component of Samba is responsible here, I don't know for which I should increase logging and for which I shouldn't. Now that I'm digging, there also seem to be some generic WERR_BADFILE DRS replication errors that our automated monitoring somehow didn't catch; and one DC apparently no longer has the DNS entries it should have, and samba_dnsupdates alternates between "FORMERR" and "GSS-TSIG unsuccessful" which apparently is only supposed to appear with the BIND9 DNS backend, which we aren't using. These are probably related, but again I have no idea where these come from or how to debug them. So how was your morning? -- Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, Systemadministrator ✉ sven.schwedas at tao.at | ☎ +43 680 301 7167 TAO Digital | Teil der TAO Beratungs- & Management GmbH Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach A8020 Graz | https://www.tao-digital.at -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20190521/069c3770/signature.sig>
L.P.H. van Belle
2019-May-21 12:37 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
Hai Sven, And still i see/think you should change some things to get a better base setup. And no its not bike shedding.... It is making a standard setup, work from there. [libdefaults] default_realm = AD.TAO.AT dns_lookup_realm = true < if you have multple REALM, else false. (default_realm = AD.TAO.AT) dns_lookup_kdc = true Checking file: /etc/nsswitch.conf passwd: files winbind group: files winbind shadow: files ( removed winbind from shadow) not used. winbind enum users = yes winbind enum groups = yes Better no, works the same, but your server is faster. #### site.conf netbios name = villach-file < in CAPS For windows/samba netbios resolving: NETBIOSNAME =! netbiosname DNS resolving : NETBIOSNAME == netbiosname REALM resolving : REALM =! realm Dnsdomain name : realm often looks like dnsdomainname but.. dnsdomainname =! REALM .. Clean up you site.conf. Make it as little as possible. You see this note from the script: Running as Unix domain member and no user.map detected. Where is you user mapping? You dont use SePrivileges? Now its not wrong and possible to run it without, but it is much more work to setup correctly for this. And.. You still on 4.5.16, yes, possible, but why do you think i make newer packages. Windows and it updates are moving fast, so samba is following fast, while debian is slow. Not that's wrong, really i preffer myself slow and good updates, but thats just not the way for samba. And this is why i build the samba packages. To keep up with samba. You cant fix all with 4.5.16, for that you need higher samba versions. I've suggested this to Debian, to make a separated line for samba that follow the main releases of samba. But, that as a no-no.., so thats why i supply these, with debian's settings. Thats also why i use distro-sambaVERSION , to keep track with samba AND windows. Now, last question, on the pc with the "unable to authenticate", any windows event id's with warning/errors? You probley looked at that already?? Or not?> Top level error I'm seeing is that since today *some* Windows > users are denied SMB access to this one member server ("Network password is > invalid"), but not all users. Worked fine before today.If you delayed your windows updates for for example 6 day, then this is logical to me. Because MS updates are on Tuesday.. Now, what if you reinstall SMB1 for these windows pc and disable autoremovement. Check with Powershel: Get-WindowsFeature FS-SMB1 If thats not the case, then you should check the attributes of the computer in the AD. this could be also due to kerberos mismatchings in AD. You can check this as followed. samba-tool computer show YOUR_COMPUTERNAME_HERE > /tmp/YOUR_COMPUTERNAME_HERE.txt egrep "dn|name|sAMAccountName|dNSHostName|distinguishedName|servicePrincipalName" < /tmp/YOUR_COMPUTERNAME_HERE.txt Safe the file or you keep quering your AD. servicePrincipalName: HOST/HOSTNAME.dnsdomain.tld is WRONG ! servicePrincipalName: HOST/HOSTNAME is correct. servicePrincipalName: HOST/hostname.dnsdomain.tld is correct ! So correct: HOST/NETBIOSNAME ( uppercase) HOST/host.fqdn ( lowercase) sAMAccountName: NETBIOSNAME$ ( uppercase) Check this if this is your case also, there are lots of reports if "unable to authenticate" or lost trust of domain.. Due to above. If a name is wrong, Open ADSIEdit. Go to the computer object. Don't hit properties of the object just right click and choose rename. More below...> -----Oorspronkelijk bericht----- > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Sven > Schwedas via samba > Verzonden: dinsdag 21 mei 2019 13:28 > Aan: samba at lists.samba.org > Onderwerp: Re: [Samba] Debugging Samba is a total PITA and > this needs to improve > > The smb.conf hasn't changed since the last three or four times I've > posted here asking for help: > > https://up.tao.at/u/samba/villach-file.txt > > Top level error I'm seeing is that since today *some* Windows > users are > denied SMB access to this one member server ("Network password is > invalid"), but not all users. Worked fine before today. > > wbinfo -p/-P work, wbinfo -a shows the same problem of some users > working, some not: Those that do work, report success with plaintext > auth, and NT_STATUS_WRONG_PASSWORD for challenge/response auth (wtf?). > Those that don't work at all, fail plaintext auth and report > NT_STATUS_INTERNAL_DB_CORRUPTION for challenge/response. Not sure if > that means anything, given that challenge/response seems to > always fail > with nonsensical error messages. All the other working member servers > also report NT_STATUS_WRONG_PASSWORD for c/r auth. > > 15 MB/s error logs were not an exaggeration, BTW, that's what > I saw when > I cranked up the logging level to 10, since the default log > level didn't > bother even reporting the logon failures at all (which should be > sensible defaults, but oh well). Since I don't know what component of > Samba is responsible here, I don't know for which I should increase > logging and for which I shouldn't.man smb.conf /log level ( + hit 5x n ) and your at the log level point. ;-) That shows this example : log level = 1 full_audit:1@/var/log/audit.log> > Now that I'm digging, there also seem to be some generic WERR_BADFILE > DRS replication errors that our automated monitoring somehow didn't > catch; and one DC apparently no longer has the DNS entries it should > have, and samba_dnsupdates alternates between "FORMERR" and "GSS-TSIG > unsuccessful" which apparently is only supposed to appear with the BIND9 > DNS backend, which we aren't using. These are probably related, but > again I have no idea where these come from or how to debug them.Date: Tue, 22 May 2018 15:44:36 +0000 - Dynamic DNS updates with GSS-TSIG against Microsoft or samba DNS servers are not working and fails with the following error: ; TSIG error ... https://bugzilla.samba.org/show_bug.cgi?id=13019 samba 4.7 and lower. You really want to try my packages.. ;-) And in your case, update steps, 4.8, and stay there if you want to switch to Buster then 4.9.5 Or move more up to 4.9 or 4.10. Or if the server is an samba only, server, upgrade to buster, but ... Prepair for that, you will hit more then you expecting to hit. Not advice, just a suggesting, im not you, i can tell whats best for you, i dont know you complete network.> > > So how was your morning?Good, thanks for asking. And in addition to Rowland, you always replies when im still typing :-p ;-)>> You need to investigate your DB problems, but just a few comments onNo, start with your resolving and hostname. This is the base and this has to be correct and having this correct, helps in reducing problems in you windows clients. And it helps if finding your problem. Greetz, Louis
Aurélien Aptel
2019-May-21 13:32 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
Sven Schwedas via samba <samba at lists.samba.org> writes:> Once again, something with Samba thirty bazillion components broke. Once > again, my choices for logging are "nothing" or "15 MB/s spread of ten > different files, because 'client authentication failed' totally needs to > be lower priority than malloc debug info". Once again, none of these > messages is actually able to convey what broke, where, why. Why is itRegarding the huge and verbose logs: I can suggest you smblog-mode[1], it's not really a solution and it's kind of made for emacs users but it makes sorting thru logs less painful. It looks like this [2]. You can filter log levels dynamically with + and -, expand/collapse messages with TAB, highlight all messages printed between receiving a request and sending its response, etc. I have another tool smbcmp [3] you can use to compare SMB captures which can be useful e.g. if something works from a machine and not from another. As a side note: I'm doing a talk on smb debugging at next sambaxp to show how these are used. 1: https://github.com/aaptel/smblog-mode 2: https://lists.samba.org/archive/samba-technical/attachments/20160315/4a04dddf/smblog3-0001.png 3: https://github.com/aaptel/smbcmp Cheers, -- Aurélien Aptel / SUSE Labs Samba Team GPG: 1839 CB5F 9F5B FB9B AA97 8C99 03C8 A49B 521B D5D3 SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnberg)
Ralph Boehme
2019-May-21 14:06 UTC
[Samba] Debugging Samba is a total PITA and this needs to improve
On 5/21/19 3:32 PM, Aurélien Aptel via samba wrote:> Sven Schwedas via samba <samba at lists.samba.org> writes: >> Once again, something with Samba thirty bazillion components broke. Once >> again, my choices for logging are "nothing" or "15 MB/s spread of ten >> different files, because 'client authentication failed' totally needs to >> be lower priority than malloc debug info". Once again, none of these >> messages is actually able to convey what broke, where, why. Why is it > > Regarding the huge and verbose logs: I can suggest you smblog-mode[1], > it's not really a solution and it's kind of made for emacs users but it > makes sorting thru logs less painful....I guess we also need wblog-mode for winbindd. :) Cheerio! -slow
Alright, I managed to jury-rig LDAP-authenticated FTP access to the file shares, which works well enough that I no longer have the whole company breathing down my neck. Sorry for being so cranky yesterday. From what we've established, it seems that there are a bunch of replication issues, maybe others: • Authentication on one member server fails for some users, but not all. There doesn't seem to be any pattern to who is and isn't affected. • Trust between PCs and the domain is randomly lost. That was handled by L1 support without telling me (they just re-joined the PCs every other week), so I wasn't aware of that yesterday • All sorts of replication issues between the various DCs:> https://up.tao.at/u/samba/graz-dc-sem.txt (FSMO role holder) > https://up.tao.at/u/samba/graz-dc-1b.txt > https://up.tao.at/u/samba/villach-dc-1a.txt > https://up.tao.at/u/samba/villach-dc-bis.txt(Unchanged from yesterday) • Some DB issues:> https://up.tao.at/u/samba/graz-dc-sem-dbcheck.txt > https://up.tao.at/u/samba/graz-dc-1b-dbcheck.txt > https://up.tao.at/u/samba/villach-dc-1a-dbcheck.txt > https://up.tao.at/u/samba/villach-dc-bis-dbcheck.txt(Unchanged from yesterday) And since we now actually have time for this, all collected info for all domain-joined servers: https://up.tao.at/u/samba/graz-dc-1b.info.txt https://up.tao.at/u/samba/graz-dc-sem.info.txt https://up.tao.at/u/samba/graz-file.info.txt https://up.tao.at/u/samba/graz-mail.info.txt https://up.tao.at/u/samba/villach-dc-1a.info.txt https://up.tao.at/u/samba/villach-dc-bis.info.txt https://up.tao.at/u/samba/villach-file.info.txt https://up.tao.at/u/samba/villach-mail.info.txt I have no idea how villach-dc-bis worked so far with that krb5.conf setup, but I guess that needs fixing. For general smb.conf stuff, the features we now still need are • rfc2307-based Unix attributes (it's where we get all UIDs/GIDs from) • ACLs are set programmatically with a script (management wants that locked down), so if there's a better way to handle those, we can just nuke the current setup and reset them all • ACLs still need to work for people SSHing (or now FTPing) into the servers with their domain accounts, not just from Windows • New features (like SePrivileges support) aren't interesting, file shares will be deprecated in favour of Cloud™ Stuff™ soon-ish anyway (let's see for how long) • Mail servers need winbind expansion and enumeration to support legacy stuff that we won't get around to fix for a while yet (the domain has like 50 users and a maximum nesting depth of 1, so performance is fine for now) With these constraints, what are the minimal smb.conf setups I need? Other todos for now: • Fix krb5.conf everywhere (remove dns_lookup_realm?) • Improve nsswitch setup I'll get to that when I know what all the smb.confs need to look like. And going back to Louis' email: On 22.05.19 10:29, L.P.H. van Belle wrote:> Yes, then it does take the hostname. Just be warned, that in such > cases the hostname max lenght is 15 chars.That used to be a problem a while back when we had longer hostnames, but shouldn't apply with the current setup. So I guess we can get rid of that too.> You need to verify all GUID of the DC's. ( and A/PTR/CNAME records ) > ldbsearch -H /var/lib/samba/private/sam.ldb '(fromServer=*CN=Windows-DC*)' --cross-ncs dnThat gives me no results on all DCs. The wiki suggests using "ldbsearch -H /var/lib/samba/private/sam.ldb '(invocationId=*)' --cross-ncs objectguid" for GUIDs, which does give me results.> Samba INTERNAL_DNS Back End - TroubleshootingThat seems to work fine, there's nothing blocking samba's internal DNS from listening on :53, and I can query it just fine.> https://wiki.samba.org/index.php/Verifying_and_Creating_a_DC_DNS_Recordvillach-dc-bis doesn't have its _msdcs record set on /any/ server. All other records are fine on all servers. I set it on graz-dc-sem as detailed in the wiki, and it replicated fine to all servers except graz-dc-1b. I restarted samba-ad-dc on that to increase the log level, and that restart alone seems to have done the trick, it now has the record as well. I also fixed villach-dc-bis Kerberos to> [libdefaults] > default_realm = AD.TAO.AT > dns_lookup_kdc = true > default_keytab_name = FILE:/etc/krb5.keytab > > [domain_realm] > .ad.tao.at = AD.TAO.AT > ad.tao.at = AD.TAO.AT > .tao.at = AD.TAO.AT > tao.at = AD.TAO.ATand restarted samba-ad-dc on it. Now, trying to full sync replicate works fine for the base partition and Configuration, but replicating ForestDnsZones or DomainDnsZones both fail with:> ERROR(<class 'samba.drs_utils.drsException'>): DsReplicaSync failed - drsException: DsReplicaSync failed (8440, 'WERR_DS_DRA_BAD_NC') > File "/usr/lib/python2.7/dist-packages/samba/netcmd/drs.py", line 368, in run > drs_utils.sendDsReplicaSync(server_bind, server_bind_handle, source_dsa_guid, NC, req_options) > File "/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line 83, in sendDsReplicaSync > raise drsException("DsReplicaSync failed %s" % estr)Replicating Schema still runs to a timeout on the sending server and a PANIC on villach-dc-bis. Log file on villach-dc-bis: https://up.tao.at/u/samba/log.samba.txt Relevant time stamps 2019/05/22 12:26:26.456202 (ForestDnsZones failure), 2019/05/22 12:26:49.027595 (DomainDnsZones failure) and 2019/05/22 12:27:03.755312 (Schema panic) -- Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, Systemadministrator ✉ sven.schwedas at tao.at | ☎ +43 680 301 7167 TAO Digital | Teil der TAO Beratungs- & Management GmbH Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach A8020 Graz | https://www.tao-digital.at -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20190522/05e97fb7/signature.sig>
Hai, Well, good that your more relax and releaved in pressure.. Apoligies accepted. ;-), i know the fealing but do understand, we are trying to help out.. As you know, it very frustration to ask the same things again, and you showing them again But now as you showed with all the configs. It is needed.. (sorry) So I had a quick look, and you up to some changes. All are fixable, no worries, but it takes a bit of time and you need to be precise/accurate. For example, https://up.tao.at/u/samba/graz-dc-1b.info.txt Hostname: graz-dc-1b DNS Domain: FQDN: graz-dc-1b ipaddress: 169.254.93.226 192.168.17.66 - Where is the DNS domain? ( i'll add warning in the script there if its missing.) Cause: Checking file: /etc/hosts 127.0.0.1 graz-dc-1b localhost - Which is wrong. - I would have expected to see: 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters 192.168.17.66 graz-dc-1b.ad.tao.at graz-dc-1b And this one, i should not see that, execpt if its a dhcp thats failing. 169.254.93.226 NetName: LINKLOCAL-RFC3927-IANA-RESERVED And if you did disable IPv6, which you attempted, at least looks like it. inet6 fe80::b8de:f4ff:fe1e:11e5/64 scope link is on the interface, but the ipv6 parts where missing in /etc/hosts. /etc/resolv.conf nameserver 192.168.17.1 Only one DC? And you have 4.. For every DC and per site, you have 2 sites correct? I suggest, per site, 2 DC of the same site, 1 DC of the remote as backup. Look at this, explanation is below it. The DC1 with fsmo. /etc/resolv.conf ( by example ) domain ad.tao.at nameserver IP_DC1_WITH_FSMO nameserver IP_DC2 nameserver IP_DC3_S2 DC2. /etc/resolv.conf ( by example ) domain ad.tao.at nameserver IP_DC1_WITH_FSMO nameserver IP_DC2 nameserver IP_DC4_S2 # Before reboot and after reboot and wait time and db replication check. #nameserver IP_DC2 #nameserver IP_DC1_WITH_FSMO #nameserver IP_DC4_S2 # one example site 2. # Before.. DC2. /etc/resolv.conf # depending on how your site is defined in AD. # search site2.tao.at ad.tao.at # search site2.tao.at # if all in same domain: search ad.tao.at nameserver IP_DC1_WITH_FSMO nameserver IP_DC1_SITE2 nameserver IP_DC2_SITE2 # After reboot and after checking the resolving. /etc/resolv.conf nameserver IP_DC1_SITE2 nameserver IP_DC2_SITE2 nameserver IP_DC1_WITH_FSMO # some options you can play with, put them in the resolv.conf but do not enable them yet. # I use bind, you are using internal DNS. #options timeout:2 #options attempts:2 #options edns0 #options single-request #options single-request-reopen Now the 2 resolv.conf examples and /etc/hosts file, should make sure your hostname and dnsdomain is correct. As shown per example this needs to be correct before we do anything else, and yes, fix this for every server. For the replication. You see, i've kept in both resolv.conf files DC1 as first, you need this for the first replication. After a reboot and at least 15 min-30 main waiting, on DC2, switch the DC1 and DC2 lines and set DC2 first. And this is for every server needed. I wont hurt if DC1 stays first, but that could overload DC1 on DNS requests. For all DC's and members, all you need: ( i install krb5-user, define the REALM and keep everythings default. So this is sufficient. ( yes only that one default_realm line, others are defaults ) /etc/krb5.conf [libdefaults] default_realm = AD.TAO.AT Except, i noticed, .tao.at = AD.TAO.AT tao.at = AD.TAO.AT That might need some more explanation why you added it. That helps me understanding your setup. SSO login with users email adresses maybe? /etc/nsswitch.conf passwd: files winbind group: files winbind shadow: files < winbind removed here. /etc/samba/smb.conf ldap ssl = start tls ldap ssl ads = yes These are not supported in the AD setup. These are for an NT4DOM/ PDC/BDC setup with ldap. And i like how you did split up the global part and the "real" server config part. One im going to use also.. So noo, not only bad thin, also good things.. I noticed also: #FIXME: Temporary to fix PHP shit ldap server require strong auth = no # explain this to me, can offlist if needed. Because here, i think you forgot to also define /etc/ldap/ldap.conf .. And i missed this in the script, i'll add this .. # /etc/ldap/ldap.conf # example, change the hostnames.. ( saved me typing ) ;-) BASE dc=ad,dc=tao,dc=at URI ldaps://dc1.ad.tao.at ldaps://rtd-dc2.... ldaps://dc3... ldaps://dc4... # and here again, first the 2 within the same site, then the 2 others. # Note, all my servers use ldaps (ssl), you can use ldap with TLS also. # TLS certificates (needed for GnuTLS) TLS_CACERT /etc/ssl/certs/ca-certificates.crt # and minimal TLS_REQCERT allow # i did see you added the certificates also with update-ca-certificates correct, if yes. # then the company cert is in : /etc/ssl/certs/ca-certificates.crt # if no, then you need to define it. Last one, [homes] msdfs root = yes msdfs proxy = \\graz-file\homes As per MS advice, \\graz-file.ad.tao.at\users You might hit a bug here because [homes] on the DC's is not really supported. Better might be [users] but this one needs planning... before you change it. You understand that.. Installed packages: Missing attr Which is a must install for the DC's. Now this was only 1 server but this is exact the same for all DC's This must be corrected before we can even look at the DB replication problem. Almost all server suffer from simular problems in the setup. I suggest start with above. That is the base. For the members, hosts resolv.conf nsswitch.conf krb5.conf all need to be fixed first. Scary, yes, i understand, start with DC1_fsmo DC. Do one at the time, check all the above, so make a checklist for this. I have more, but start here, doing this will give you the ability to resync the AD-DB from DC1 to DC2 in site1. And from there we can work to the member fixes. Most probly some problem will disappear after these fixed, and sure you get some back, but these you get only once. Because now the base setup is correct, and will never change again. My changes on my DC's ( since wheeze and samba 4.1.x ), only 3 lines in smb.conf due to samba upgrades. And one in bind configs due to changes in the path of bind_dlz. Sorry, its a bit of work, i know, but put in one day now, and your able to set it and forget it and your future ready.. Now, im back to building the new 4.10.4 packages. For sofar, Greetz, Louis> -----Oorspronkelijk bericht----- > Van: Sven Schwedas [mailto:sven.schwedas at tao.at] > Verzonden: woensdag 22 mei 2019 12:42 > Aan: samba at lists.samba.org > CC: L.P.H. van Belle > Onderwerp: Various AD issues; summary > > Alright, I managed to jury-rig LDAP-authenticated FTP access > to the file > shares, which works well enough that I no longer have the > whole company > breathing down my neck. Sorry for being so cranky yesterday. > > > > From what we've established, it seems that there are a bunch of > replication issues, maybe others: > > ? Authentication on one member server fails for some users, > but not all. > There doesn't seem to be any pattern to who is and isn't affected.> > ? Trust between PCs and the domain is randomly lost. That was > handled by > L1 support without telling me (they just re-joined the PCs every other > week), so I wasn't aware of that yesterday > > ? All sorts of replication issues between the various DCs: > > > https://up.tao.at/u/samba/graz-dc-sem.txt (FSMO role holder) > > https://up.tao.at/u/samba/graz-dc-1b.txt > > https://up.tao.at/u/samba/villach-dc-1a.txt > > https://up.tao.at/u/samba/villach-dc-bis.txt > > (Unchanged from yesterday) > > ? Some DB issues: > > > https://up.tao.at/u/samba/graz-dc-sem-dbcheck.txt > > https://up.tao.at/u/samba/graz-dc-1b-dbcheck.txt > > https://up.tao.at/u/samba/villach-dc-1a-dbcheck.txt > > https://up.tao.at/u/samba/villach-dc-bis-dbcheck.txt > > (Unchanged from yesterday) > > And since we now actually have time for this, all collected > info for all > domain-joined servers: > > https://up.tao.at/u/samba/graz-dc-1b.info.txt > https://up.tao.at/u/samba/graz-dc-sem.info.txt > https://up.tao.at/u/samba/graz-file.info.txt > https://up.tao.at/u/samba/graz-mail.info.txt > https://up.tao.at/u/samba/villach-dc-1a.info.txt > https://up.tao.at/u/samba/villach-dc-bis.info.txt > https://up.tao.at/u/samba/villach-file.info.txt > https://up.tao.at/u/samba/villach-mail.info.txt > > I have no idea how villach-dc-bis worked so far with that krb5.conf > setup, but I guess that needs fixing. > > For general smb.conf stuff, the features we now still need are > > ? rfc2307-based Unix attributes (it's where we get all UIDs/GIDs from) > ? ACLs are set programmatically with a script (management wants that > locked down), so if there's a better way to handle those, we can just > nuke the current setup and reset them all > ? ACLs still need to work for people SSHing (or now FTPing) into the > servers with their domain accounts, not just from Windows > ? New features (like SePrivileges support) aren't interesting, file > shares will be deprecated in favour of Cloud? Stuff? soon-ish anyway > (let's see for how long) > ? Mail servers need winbind expansion and enumeration to > support legacy > stuff that we won't get around to fix for a while yet (the domain has > like 50 users and a maximum nesting depth of 1, so performance is fine > for now) > > With these constraints, what are the minimal smb.conf setups I need? > > Other todos for now: > > ? Fix krb5.conf everywhere (remove dns_lookup_realm?) > ? Improve nsswitch setup > > I'll get to that when I know what all the smb.confs need to look like. > > And going back to Louis' email: > > On 22.05.19 10:29, L.P.H. van Belle wrote: > > Yes, then it does take the hostname. Just be warned, that in such > > cases the hostname max lenght is 15 chars. > > That used to be a problem a while back when we had longer > hostnames, but > shouldn't apply with the current setup. So I guess we can get rid of > that too. > > > You need to verify all GUID of the DC's. ( and A/PTR/CNAME > records ) > > ldbsearch -H /var/lib/samba/private/sam.ldb > '(fromServer=*CN=Windows-DC*)' --cross-ncs dn > > That gives me no results on all DCs. The wiki suggests using > "ldbsearch > -H /var/lib/samba/private/sam.ldb '(invocationId=*)' --cross-ncs > objectguid" for GUIDs, which does give me results. > > > Samba INTERNAL_DNS Back End - Troubleshooting > > That seems to work fine, there's nothing blocking samba's internal DNS > from listening on :53, and I can query it just fine. > > > > https://wiki.samba.org/index.php/Verifying_and_Creating_a_DC_D > NS_Record > > villach-dc-bis doesn't have its _msdcs record set on /any/ server. All > other records are fine on all servers. > > I set it on graz-dc-sem as detailed in the wiki, and it > replicated fine > to all servers except graz-dc-1b. I restarted samba-ad-dc on that to > increase the log level, and that restart alone seems to have done the > trick, it now has the record as well. > > I also fixed villach-dc-bis Kerberos to > > [libdefaults] > > default_realm = AD.TAO.AT > > dns_lookup_kdc = true > > default_keytab_name = FILE:/etc/krb5.keytab > > > > [domain_realm] > > .ad.tao.at = AD.TAO.AT > > ad.tao.at = AD.TAO.AT > > .tao.at = AD.TAO.AT > > tao.at = AD.TAO.AT > > and restarted samba-ad-dc on it. > > Now, trying to full sync replicate works fine for the base > partition and > Configuration, but replicating ForestDnsZones or DomainDnsZones both > fail with: > > > ERROR(<class 'samba.drs_utils.drsException'>): > DsReplicaSync failed - drsException: DsReplicaSync failed > (8440, 'WERR_DS_DRA_BAD_NC') > > File > "/usr/lib/python2.7/dist-packages/samba/netcmd/drs.py", line > 368, in run > > drs_utils.sendDsReplicaSync(server_bind, > server_bind_handle, source_dsa_guid, NC, req_options) > > File > "/usr/lib/python2.7/dist-packages/samba/drs_utils.py", line > 83, in sendDsReplicaSync > > raise drsException("DsReplicaSync failed %s" % estr) > > Replicating Schema still runs to a timeout on the sending server and a > PANIC on villach-dc-bis. > > Log file on villach-dc-bis: https://up.tao.at/u/samba/log.samba.txt > > Relevant time stamps 2019/05/22 12:26:26.456202 (ForestDnsZones > failure), 2019/05/22 12:26:49.027595 (DomainDnsZones failure) and > 2019/05/22 12:27:03.755312 (Schema panic) > > -- > Mit freundlichen Grüßen, / Best Regards, > Sven Schwedas, Systemadministrator > ??? sven.schwedas at tao.at | ??? +43 680 301 7167 > TAO Digital | Teil der TAO Beratungs- & Management GmbH > Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach > A8020 Graz | https://www.tao-digital.at > >
On 22.05.19 15:31, L.P.H. van Belle wrote:> Hai, > > Well, good that your more relax and releaved in pressure.. > Apoligies accepted. ;-), i know the fealing but do understand, we are trying to help out.. > > As you know, it very frustration to ask the same things again, and you showing them again > But now as you showed with all the configs. It is needed.. (sorry)Yeah, now that I have the time for it. That's why I posted them all.> So I had a quick look, and you up to some changes. All are fixable, no worries, > but it takes a bit of time and you need to be precise/accurate. > > For example, > > https://up.tao.at/u/samba/graz-dc-1b.info.txt > Hostname: graz-dc-1b > DNS Domain: > FQDN: graz-dc-1b > ipaddress: 169.254.93.226 192.168.17.66 > - Where is the DNS domain? ( i'll add warning in the script there if its missing.)Fixed.> Cause: Checking file: /etc/hosts > > 127.0.0.1 graz-dc-1b localhost > - Which is wrong. > > - I would have expected to see: > 127.0.0.1 localhost > > ::1 localhost ip6-localhost ip6-loopback > ff02::1 ip6-allnodes > ff02::2 ip6-allrouters > > 192.168.17.66 graz-dc-1b.ad.tao.at graz-dc-1bFixed.> And this one, i should not see that, execpt if its a dhcp thats failing. > 169.254.93.226 NetName: LINKLOCAL-RFC3927-IANA-RESERVED > > And if you did disable IPv6, which you attempted, at least looks like it. > inet6 fe80::b8de:f4ff:fe1e:11e5/64 scope link is on the interface, but the ipv6 parts where missing in /etc/hosts.No idea what systemd-networkd is even doing there. None of this is intended… Fixed now.> /etc/resolv.conf > nameserver 192.168.17.1 > Only one DC? And you have 4..For the record, that's not a DC, that's a dnsmasq on the firewall. Requests for ad.tao.at and internal range RDNS requests are forwarded to the samba servers, the rest goes to external servers. DNSMasq is there to provide stuff not related to the samba domain, which is mostly needed for client PCs. So switching over DCs and member servers to using AD DCs directly is easy, but client PCs will need to remain on DNSMasq. From what I've seen, all AD related queries resolve fine with it, too.> For every DC and per site, you have 2 sites correct?2 physical sites (192.168.17/24 and 192.168.16/24) in different subnets, but they're organized as a single logical site in AD and VPN'd together via default gateway.> I suggest, per site, 2 DC of the same site, 1 DC of the remote as backup. > Look at this, explanation is below it. The DC1 with fsmo. > > Now the 2 resolv.conf examples and /etc/hosts file, should make sure your hostname and dnsdomain is correct. > As shown per example this needs to be correct before we do anything else, and yes, fix this for every server.Should be fixed now.> For the replication. > You see, i've kept in both resolv.conf files DC1 as first, you need this for the first replication. > After a reboot and at least 15 min-30 main waiting, on DC2, switch the DC1 and DC2 lines and set DC2 first. > And this is for every server needed. I wont hurt if DC1 stays first, but that could overload DC1 on DNS requests.Shouldn't be an issue with how few clients are involved here, but I'll keep an eye on it.> For all DC's and members, all you need: ( i install krb5-user, define the REALM and keep everythings default. > So this is sufficient. ( yes only that one default_realm line, others are defaults ) > > /etc/krb5.conf > [libdefaults] > default_realm = AD.TAO.AT > > Except, i noticed, > .tao.at = AD.TAO.AT > tao.at = AD.TAO.AT > That might need some more explanation why you added it. > That helps me understanding your setup. SSO login with users email adresses maybe?Legacy setup had our domain on tao.at directly, I think we needed that to smooth over the transition. But that was years ago and shouldn't be needed any more.> /etc/nsswitch.conf > passwd: files winbind > group: files winbind > shadow: files < winbind removed here.Fixed too, probably.> /etc/samba/smb.conf > ldap ssl = start tls > ldap ssl ads = yes > These are not supported in the AD setup. These are for an NT4DOM/ PDC/BDC setup with ldap.Might want to actually document that somewhere, this isn't clear from either wiki nor manpage.> And i like how you did split up the global part and the "real" server config part. > One im going to use also.. So noo, not only bad thin, also good things.. > > I noticed also: > #FIXME: Temporary to fix PHP shit > ldap server require strong auth = no > > # explain this to me, can offlist if needed.Some horribly obsolete PHP5 LDAP library breaks with SSL, as a workaround we're SSH tunneling it until we can kill it. Which *hopefully* should be sometimes this month anyway, so no point in fixing it now.> Because here, i think you forgot to also define /etc/ldap/ldap.conf > .. And i missed this in the script, i'll add this ..Added as well.> Last one, > [homes] > msdfs root = yes > msdfs proxy = \\graz-file\homes > > As per MS advice, \\graz-file.ad.tao.at\users > > > You might hit a bug here because [homes] on the DC's is not really supported. > Better might be [users] but this one needs planning... before you change it. > You understand that..I don't think we're even using it nowadays, so I've gutted it from all DCs.> Installed packages: > Missing attr > Which is a must install for the DC's.Should be fixed too.> Now this was only 1 server but this is exact the same for all DC's > This must be corrected before we can even look at the DB replication problem. > Almost all server suffer from simular problems in the setup. > > I suggest start with above. That is the base. > For the members, hosts resolv.conf nsswitch.conf krb5.conf all need to be fixed first. > Scary, yes, i understand, start with DC1_fsmo DC. > > Do one at the time, check all the above, so make a checklist for this.They should be all modified accordingly now: https://up.tao.at/u/samba/graz-file.info2.txt https://up.tao.at/u/samba/graz-mail.info2.txt https://up.tao.at/u/samba/graz-dc-sem.info2.txt https://up.tao.at/u/samba/graz-dc-1b.info2.txt https://up.tao.at/u/samba/villach-file.info2.txt https://up.tao.at/u/samba/villach-mail.info2.txt https://up.tao.at/u/samba/villach-dc-1a.info2.txt https://up.tao.at/u/samba/villach-dc-bis.info2.txt And now that business hours are over, also all restarted. I'll let them do their thing overnight and check up on replication status tomorrow morning. -- Mit freundlichen Grüßen, / Best Regards, Sven Schwedas, Systemadministrator ✉ sven.schwedas at tao.at | ☎ +43 680 301 7167 TAO Digital | Teil der TAO Beratungs- & Management GmbH Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach A8020 Graz | https://www.tao-digital.at -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 659 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20190522/e477c933/signature.sig>
Hi Sven, Ok, before i back to my packaging, i reviewed the changes. You still missed a bit but it looks much better already, your completely there, but we will get there. And you really need to be more precise, why im telling that... Well the review. ( thats not all, im still reviewing the base only) ... https://up.tao.at/u/samba/graz-file.info2.txt Checking file: /etc/nsswitch.conf passwd: files winbind winbind group: files winbind winbind shadow: files winbind Uhmm.. If your using puppet or ansible, then i suggest go over these files once manualy. https://up.tao.at/u/samba/graz-mail.info2.txt Missing packages. : apt install acl samba-dsdb-modules samba-vfs-modules Yes, its possible to run without these, but for a domain member better to install them. /etc/hosts # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters 127.0.0.1 localhost 192.168.17.79 graz-mail.ad.tao.at graz-mail mail.graz.tao.at 192.168.17.58 database.local 192.168.16.209 mail.villach.tao.at Now the last 2 entries, if you resolving is correct, then the last too lines should not be needed. But, its not wrong, as long as the resolves also on the other servers where its needed. https://up.tao.at/u/samba/graz-dc-sem.info2.txt You are probley having a good reason for it. template shell = /bin/zsh Your (some) other server have template shell = /bin/bash https://up.tao.at/u/samba/graz-dc-1b.info2.txt Debian 9.8 the others are at 9.9 ( apt dist-upgrade) # /etc/nsswitch.conf passwd: files winbind group: files winbind shadow: files winbind << ?? You missed this one. gshadow: files https://up.tao.at/u/samba/villach-dc-1a.info2.txt This computer is running Debian 9.0 .. You missed a lot of upgrades here. Missing acl package Well, it looks much better already. Now once all above is done, then next is. Compair your installed packages, you can do yourself. Now, this is a diff of you packages installed on the DC's. What do you see here... I see alot of differences. diff graz-dc-1b.info2.txt graz-dc-sem.info2.txt 11,21c11,24 < ii libnss-winbind:amd64 2:4.5.16+dfsg-1 amd64 Samba nameservice integration plugins < ii libpam-winbind:amd64 2:4.5.16+dfsg-1 amd64 Windows domain authentication integration plugin < ii libwbclient0:amd64 2:4.5.16+dfsg-1 amd64 Samba winbind client library < ii python-samba 2:4.5.16+dfsg-1 amd64 Python bindings for Samba < ii samba 2:4.5.16+dfsg-1 amd64 SMB/CIFS file, print, and login server for Unix < ii samba-common 2:4.5.16+dfsg-1 all common files used by both the Samba server and client < ii samba-common-bin 2:4.5.16+dfsg-1 amd64 Samba common files used by both the server and the client < ii samba-dsdb-modules 2:4.5.16+dfsg-1 amd64 Samba Directory Services Database < ii samba-libs:amd64 2:4.5.16+dfsg-1 amd64 Samba core libraries < ii samba-vfs-modules 2:4.5.16+dfsg-1 amd64 Samba Virtual FileSystem plugins < ii winbind 2:4.5.16+dfsg-1 amd64 service to resolve user and group information from Windows NT servers ---> ii libnss-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba nameservice integration plugins > ii libpam-krb5:amd64 4.7-4 amd64 PAM module for MIT Kerberos > ii libpam-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Windows domain authentication integration plugin > ii libsmbclient:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 shared library for communication with SMB/CIFS servers > ii libwbclient0:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba winbind client library > ii python-samba 2:4.5.16+dfsg-1+deb9u2 amd64 Python bindings for Samba > ii samba 2:4.5.16+dfsg-1+deb9u2 amd64 SMB/CIFS file, print, and login server for Unix > ii samba-common 2:4.5.16+dfsg-1+deb9u2 all common files used by both the Samba server and client > ii samba-common-bin 2:4.5.16+dfsg-1+deb9u2 amd64 Samba common files used by both the server and the client > ii samba-dsdb-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Directory Services Database > ii samba-libs:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba core libraries > ii samba-vfs-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Virtual FileSystem plugins > ii smbclient 2:4.5.16+dfsg-1+deb9u2 amd64 command-line SMB/CIFS clients for Unix > ii winbind 2:4.5.16+dfsg-1+deb9u2 amd64 service to resolve user and group information from Windows NT serversdiff graz-dc-sem.info2.txt villach-dc-1a.info2.txt 2,24c2,17 < ii acl 2.2.52-3+b1 amd64 Access control list utilities < ii attr 1:2.4.47-2+b2 amd64 Utilities for manipulating filesystem extended attributes < ii krb5-config 2.6 all Configuration files for Kerberos Version 5 < ii krb5-user 1.15-1+deb9u1 amd64 basic programs to authenticate using MIT Kerberos < ii libacl1:amd64 2.2.52-3+b1 amd64 Access control list shared library < ii libattr1:amd64 1:2.4.47-2+b2 amd64 Extended attribute shared library < ii libgssapi-krb5-2:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries - krb5 GSS-API Mechanism < ii libkrb5-3:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries < ii libkrb5support0:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries - Support library < ii libnss-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba nameservice integration plugins < ii libpam-krb5:amd64 4.7-4 amd64 PAM module for MIT Kerberos < ii libpam-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Windows domain authentication integration plugin < ii libsmbclient:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 shared library for communication with SMB/CIFS servers < ii libwbclient0:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba winbind client library < ii python-samba 2:4.5.16+dfsg-1+deb9u2 amd64 Python bindings for Samba < ii samba 2:4.5.16+dfsg-1+deb9u2 amd64 SMB/CIFS file, print, and login server for Unix < ii samba-common 2:4.5.16+dfsg-1+deb9u2 all common files used by both the Samba server and client < ii samba-common-bin 2:4.5.16+dfsg-1+deb9u2 amd64 Samba common files used by both the server and the client < ii samba-dsdb-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Directory Services Database < ii samba-libs:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba core libraries < ii samba-vfs-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Virtual FileSystem plugins < ii smbclient 2:4.5.16+dfsg-1+deb9u2 amd64 command-line SMB/CIFS clients for Unix < ii winbind 2:4.5.16+dfsg-1+deb9u2 amd64 service to resolve user and group information from Windows NT servers ---> ii attr 1:2.4.47-2+b2 amd64 Utilities for manipulating filesystem extended attributes > ii krb5-config 2.6 all Configuration files for Kerberos Version 5 > ii krb5-user 1.15-1 amd64 basic programs to authenticate using MIT Kerberos > ii libacl1:amd64 2.2.52-3+b1 amd64 Access control list shared library > ii libattr1:amd64 1:2.4.47-2+b2 amd64 Extended attribute shared library > ii libgssapi-krb5-2:amd64 1.15-1 amd64 MIT Kerberos runtime libraries - krb5 GSS-API Mechanism > ii libkrb5-3:amd64 1.15-1 amd64 MIT Kerberos runtime libraries > ii libkrb5support0:amd64 1.15-1 amd64 MIT Kerberos runtime libraries - Support library > ii libwbclient0:amd64 2:4.5.6+dfsg-1 amd64 Samba winbind client library > ii python-samba 2:4.5.6+dfsg-1 amd64 Python bindings for Samba > ii samba 2:4.5.6+dfsg-1 amd64 SMB/CIFS file, print, and login server for Unix > ii samba-common 2:4.5.6+dfsg-1 all common files used by both the Samba server and client > ii samba-common-bin 2:4.5.6+dfsg-1 amd64 Samba common files used by both the server and the client > ii samba-dsdb-modules 2:4.5.6+dfsg-1 amd64 Samba Directory Services Database > ii samba-libs:amd64 2:4.5.6+dfsg-1 amd64 Samba core libraries > ii winbind 2:4.5.6+dfsg-1 amd64 service to resolve user and group information from Windows NT serversdiff villach-dc-1a.info2.txt villach-dc-bis.info2.txt 2,17c2,24 < ii attr 1:2.4.47-2+b2 amd64 Utilities for manipulating filesystem extended attributes < ii krb5-config 2.6 all Configuration files for Kerberos Version 5 < ii krb5-user 1.15-1 amd64 basic programs to authenticate using MIT Kerberos < ii libacl1:amd64 2.2.52-3+b1 amd64 Access control list shared library < ii libattr1:amd64 1:2.4.47-2+b2 amd64 Extended attribute shared library < ii libgssapi-krb5-2:amd64 1.15-1 amd64 MIT Kerberos runtime libraries - krb5 GSS-API Mechanism < ii libkrb5-3:amd64 1.15-1 amd64 MIT Kerberos runtime libraries < ii libkrb5support0:amd64 1.15-1 amd64 MIT Kerberos runtime libraries - Support library < ii libwbclient0:amd64 2:4.5.6+dfsg-1 amd64 Samba winbind client library < ii python-samba 2:4.5.6+dfsg-1 amd64 Python bindings for Samba < ii samba 2:4.5.6+dfsg-1 amd64 SMB/CIFS file, print, and login server for Unix < ii samba-common 2:4.5.6+dfsg-1 all common files used by both the Samba server and client < ii samba-common-bin 2:4.5.6+dfsg-1 amd64 Samba common files used by both the server and the client < ii samba-dsdb-modules 2:4.5.6+dfsg-1 amd64 Samba Directory Services Database < ii samba-libs:amd64 2:4.5.6+dfsg-1 amd64 Samba core libraries < ii winbind 2:4.5.6+dfsg-1 amd64 service to resolve user and group information from Windows NT servers ---> ii acl 2.2.52-3+b1 amd64 Access control list utilities > ii attr 1:2.4.47-2+b2 amd64 Utilities for manipulating filesystem extended attributes > ii krb5-config 2.6 all Configuration files for Kerberos Version 5 > ii krb5-locales 1.15-1+deb9u1 all internationalization support for MIT Kerberos > ii krb5-user 1.15-1+deb9u1 amd64 basic programs to authenticate using MIT Kerberos > ii libacl1:amd64 2.2.52-3+b1 amd64 Access control list shared library > ii libattr1:amd64 1:2.4.47-2+b2 amd64 Extended attribute shared library > ii libgssapi-krb5-2:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries - krb5 GSS-API Mechanism > ii libkrb5-3:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries > ii libkrb5support0:amd64 1.15-1+deb9u1 amd64 MIT Kerberos runtime libraries - Support library > ii libnss-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba nameservice integration plugins > ii libpam-winbind:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Windows domain authentication integration plugin > ii libsmbclient:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 shared library for communication with SMB/CIFS servers > ii libwbclient0:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba winbind client library > ii python-samba 2:4.5.16+dfsg-1+deb9u2 amd64 Python bindings for Samba > ii samba 2:4.5.16+dfsg-1+deb9u2 amd64 SMB/CIFS file, print, and login server for Unix > ii samba-common 2:4.5.16+dfsg-1+deb9u2 all common files used by both the Samba server and client > ii samba-common-bin 2:4.5.16+dfsg-1+deb9u2 amd64 Samba common files used by both the server and the client > ii samba-dsdb-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Directory Services Database > ii samba-libs:amd64 2:4.5.16+dfsg-1+deb9u2 amd64 Samba core libraries > ii samba-vfs-modules 2:4.5.16+dfsg-1+deb9u2 amd64 Samba Virtual FileSystem plugins > ii smbclient 2:4.5.16+dfsg-1+deb9u2 amd64 command-line SMB/CIFS clients for Unix > ii winbind 2:4.5.16+dfsg-1+deb9u2 amd64 service to resolve user and group information from Windows NT serversFor all DCs you need to fix/align the packages. apt-get install samba samba-libs samba-vfs-modules samba-dsdb-modules winbind ldb-tools acl attr Optional but adviced: (sso) ssh logins: apt-get install libnss-winbind libpam-winbind libpam-krb5 Optional but adviced: Client tools: apt-get install smbclient And your probely are missing this file on most of your servers. one i can add also to my script. /usr/share/pam-configs/winbind echo "Name: Winbind NT/Active Directory authentication Default: yes Priority: 192 Auth-Type: Primary Auth: [success=end default=ignore] pam_winbind.so krb5_auth krb5_ccache_type=FILE cached_login try_first_pass Auth-Initial: [success=end default=ignore] pam_winbind.so krb5_auth krb5_ccache_type=FILE cached_login Account-Type: Primary Account: [success=end new_authtok_reqd=done default=ignore] pam_winbind.so Password-Type: Primary Password: [success=end default=ignore] pam_winbind.so try_authtok try_first_pass Password-Initial: [success=end default=ignore] pam_winbind.so Session-Type: Additional Session: optional pam_winbind.so " > /usr/share/pam-configs/winbind And enable/disable it with: pam-auth-update If needed, and you probely need it for ssh logins, select/enable winbind. After this is done on all the DC's then i have 2 more things. How are you replication sysvol? How are you syncing the time between all DC'.s ( and again need to add that also to the script. ) Here i have only one advice. Make sure the time sources on all DCs are the same. And ntp.pool.org is not the same, textwise yes, ip wise no. I have seen more then 5 min differnces in the past. Here i suggest, use the NTP of you internet provider. What i now suggest, you have these files, as in these :XXXXX.info2.txt for all servers. Use these and diff between the servers. Now beside the above changes which need to be done first. Next will be. More standardization of you smb.conf for all servers. Your setup with the includes are good, but the settings are a bit off. And with off i mean, not wrong, but different, but this can cause problems. SO the more these are the same the less problems. And, keep site/server specific differences in these site includes, thats good. And some comments below in between.> -----Oorspronkelijk bericht----- > Van: Sven Schwedas [mailto:sven.schwedas at tao.at] > Verzonden: woensdag 22 mei 2019 17:53 > Aan: L.P.H. van Belle; samba at lists.samba.org > Onderwerp: Re: Various AD issues; summary > > > > On 22.05.19 15:31, L.P.H. van Belle wrote: > > Hai, > > > > Well, good that your more relax and releaved in pressure.. > > Apoligies accepted. ;-), i know the fealing but do > understand, we are trying to help out.. > > > > As you know, it very frustration to ask the same things > again, and you showing them again > > But now as you showed with all the configs. It is needed.. (sorry) > > Yeah, now that I have the time for it. That's why I posted them all. > > > So I had a quick look, and you up to some changes. All are > fixable, no worries, > > but it takes a bit of time and you need to be precise/accurate. > > > > For example, > > > > https://up.tao.at/u/samba/graz-dc-1b.info.txt > > Hostname: graz-dc-1b > > DNS Domain: > > FQDN: graz-dc-1b > > ipaddress: 169.254.93.226 192.168.17.66 > > - Where is the DNS domain? ( i'll add warning in the > script there if its missing.) > > Fixed. > > > Cause: Checking file: /etc/hosts > > > > 127.0.0.1 graz-dc-1b localhost > > - Which is wrong. > > > > - I would have expected to see: > > 127.0.0.1 localhost > > > > ::1 localhost ip6-localhost ip6-loopback > > ff02::1 ip6-allnodes > > ff02::2 ip6-allrouters > > > > 192.168.17.66 graz-dc-1b.ad.tao.at graz-dc-1b > > Fixed. > > > And this one, i should not see that, execpt if its a dhcp > thats failing. > > 169.254.93.226 NetName: LINKLOCAL-RFC3927-IANA-RESERVED > > > > And if you did disable IPv6, which you attempted, at least > looks like it. > > inet6 fe80::b8de:f4ff:fe1e:11e5/64 scope link is on the > interface, but the ipv6 parts where missing in /etc/hosts. > > No idea what systemd-networkd is even doing there. None of this is > intended? Fixed now. > > > /etc/resolv.conf > > nameserver 192.168.17.1 > > Only one DC? And you have 4.. > > For the record, that's not a DC, that's a dnsmasq on the firewall. > Requests for ad.tao.at and internal range RDNS requests are > forwarded to > the samba servers, the rest goes to external servers. > > DNSMasq is there to provide stuff not related to the samba > domain, which > is mostly needed for client PCs. So switching over DCs and member > servers to using AD DCs directly is easy, but client PCs will need to > remain on DNSMasq. From what I've seen, all AD related queries resolve > fine with it, too.What you her is fine, but resolv.conf on the DCs need to resolve to the DC's. Then you can add the ip of the dnsmask forwarder in the smb.conf Now, these are different also.> > > For every DC and per site, you have 2 sites correct? > > 2 physical sites (192.168.17/24 and 192.168.16/24) in > different subnets, > but they're organized as a single logical site in AD and > VPN'd together via default gateway.I've dont same here.. I have one site but 4 subdomains and 6 subnets. Now, that VPN part, keep that in mind, after all things are done i have questions about that one also. But lets not do that all at once.> > > I suggest, per site, 2 DC of the same site, 1 DC of the > remote as backup. > > Look at this, explanation is below it. The DC1 with fsmo. > > > > Now the 2 resolv.conf examples and /etc/hosts file, should > make sure your hostname and dnsdomain is correct. > > As shown per example this needs to be correct before we do > anything else, and yes, fix this for every server. > > Should be fixed now. > > > For the replication. > > You see, i've kept in both resolv.conf files DC1 as first, > you need this for the first replication. > > After a reboot and at least 15 min-30 main waiting, on DC2, > switch the DC1 and DC2 lines and set DC2 first. > > And this is for every server needed. I wont hurt if DC1 > stays first, but that could overload DC1 on DNS requests. > > Shouldn't be an issue with how few clients are involved here, but I'll > keep an eye on it.The dnsmask should help in avoiding that.> > > For all DC's and members, all you need: ( i install > krb5-user, define the REALM and keep everythings default. > > So this is sufficient. ( yes only that one default_realm > line, others are defaults ) > > > > /etc/krb5.conf > > [libdefaults] > > default_realm = AD.TAO.AT > > > > Except, i noticed, > > .tao.at = AD.TAO.AT > > tao.at = AD.TAO.AT > > That might need some more explanation why you added it. > > That helps me understanding your setup. SSO login with > users email adresses maybe? > > Legacy setup had our domain on tao.at directly, I think we needed that > to smooth over the transition. But that was years ago and shouldn't be > needed any more.Good, verify it and if its not needed remove it.> > > /etc/nsswitch.conf > > passwd: files winbind > > group: files winbind > > shadow: files < winbind removed here. > > Fixed too, probably. > > > /etc/samba/smb.conf > > ldap ssl = start tls > > ldap ssl ads = yes > > These are not supported in the AD setup. These are for an > NT4DOM/ PDC/BDC setup with ldap. > > Might want to actually document that somewhere, this isn't clear from > either wiki nor manpage... I think its "not clear" but work is done/will be done to improve that.> > > And i like how you did split up the global part and the > "real" server config part. > > One im going to use also.. So noo, not only bad thin, also > good things.. > > > > I noticed also: > > #FIXME: Temporary to fix PHP shit > > ldap server require strong auth = no > > > > # explain this to me, can offlist if needed. > > Some horribly obsolete PHP5 LDAP library breaks with SSL, as a > workaround we're SSH tunneling it until we can kill it. Which > *hopefully* should be sometimes this month anyway, so no > point in fixing it now.Hm, your using debian stretch, then why are you useing jessie's php5? Old web app that needed php5? Stretch its default is php7.0 One that might help fixing that php5-ldap bug is, use the surg.org packages. https://tecadmin.net/install-php-debian-9-stretch/ Just a suggestion, you can run php5.6 7.0 7.1 7.2 7.3 on stretch. It works the same, but it might fix the bug in 5.6. Just... DONT install that now, first fix all samba things.> > > Because here, i think you forgot to also define /etc/ldap/ldap.conf > > .. And i missed this in the script, i'll add this .. > > Added as well. > > > Last one, > > [homes] > > msdfs root = yes > > msdfs proxy = \\graz-file\homes > > > > As per MS advice, \\graz-file.ad.tao.at\users > > > > > > You might hit a bug here because [homes] on the DC's is not > really supported. > > Better might be [users] but this one needs planning... > before you change it. > > You understand that.. > > I don't think we're even using it nowadays, so I've gutted it > from all DCs. > > > Installed packages: > > Missing attr > > Which is a must install for the DC's. > > Should be fixed too. > > > Now this was only 1 server but this is exact the same for all DC's > > This must be corrected before we can even look at the DB > replication problem. > > Almost all server suffer from simular problems in the setup. > > > > I suggest start with above. That is the base. > > For the members, hosts resolv.conf nsswitch.conf krb5.conf > all need to be fixed first. > > Scary, yes, i understand, start with DC1_fsmo DC. > > > > Do one at the time, check all the above, so make a > checklist for this. > > They should be all modified accordingly now: > > https://up.tao.at/u/samba/graz-file.info2.txt > https://up.tao.at/u/samba/graz-mail.info2.txt > https://up.tao.at/u/samba/graz-dc-sem.info2.txt > https://up.tao.at/u/samba/graz-dc-1b.info2.txt > https://up.tao.at/u/samba/villach-file.info2.txt > https://up.tao.at/u/samba/villach-mail.info2.txt > https://up.tao.at/u/samba/villach-dc-1a.info2.txt > https://up.tao.at/u/samba/villach-dc-bis.info2.txt > > And now that business hours are over, also all restarted. > I'll let them > do their thing overnight and check up on replication status tomorrow > morning. > > -- > Mit freundlichen Grüßen, / Best Regards, > Sven Schwedas, Systemadministrator > ??? sven.schwedas at tao.at | ??? +43 680 301 7167 > TAO Digital | Teil der TAO Beratungs- & Management GmbH > Lendplatz 45 | FN 213999f/Klagenfurt, FB-Gericht Villach > A8020 Graz | https://www.tao-digital.at > >
Maybe Matching Threads
- Various AD issues; summary
- Debugging Samba is a total PITA and this needs to improve
- Server GC/name.dom/dom is not registered with our KDC: Miscellaneous failure (see text): Server (GC/name/dom@DOM) unknown
- Winbind error "Could not fetch our SID - did we join?"
- Server GC/name.dom/dom is not registered with our KDC: Miscellaneous failure (see text): Server (GC/name/dom@DOM) unknown