Hi Martin It seems as though, when I go from `clustering = no` to `clustering = yes`, if I do a domain join, it will fail. However, if I do a `systemctl restart ctdb` (knowing full well it will fail every time), if after this I add a sleep(15), then do a domain join, then do a `systemctl restart ctdb`, then the join will have worked, AND CTDB will start properly. So in a nutshell, in Ansible, - do all the samba setup without clustering on, even winbind setup; verify it works - do all the ctdb setup and turn clustering on, but we must again domain-join, but only after having run restart-ctdb once first, then after the join, do another restart-ctdb Only then does the system come to a stable point. This appears to be the only way to have a repeatable deployment process of CTDB over multiple regions globally. Any thoughts or recommendations? Bob On Thu, Oct 1, 2020 at 9:35 AM Robert Buck <robert.buck at som.com> wrote:> And more information, wondering about DNS issues or DC issues... > > # wbinfo --ping-dc > > checking the NETLOGON for domain[MYDOMAINNAME] dc connection to "" failed > > failed to call wbcPingDc: WBC_ERR_DOMAIN_NOT_FOUND > > On Thu, Oct 1, 2020 at 9:21 AM Robert Buck <robert.buck at som.com> wrote: > >> Martin, >> >> Here you go, >> >> # echo "mypassword" | net --no-dns-updates -U service-account-name ads >> testjoin domain.local >> >> kerberos_kinit_password NETBIOS_NAME$@DOMAIN.LOCAL failed: Client not >> found in Kerberos database >> >> Join to domain is not valid: The name provided is not a properly formed >> account name. >> >> On Wed, Sep 30, 2020 at 9:34 PM Martin Schwenke <martin at meltin.net> >> wrote: >> >>> Hi Bob, >>> >>> On Wed, 30 Sep 2020 08:59:41 -0400, Robert Buck <robert.buck at som.com> >>> wrote: >>> >>> > [...] >>> > Sep 30 12:58:25 euw2-samba-server-c21-01 winbindd[484378]: * Could not >>> > fetch our SID - did we join?* >>> > >>> > Sep 30 12:58:25 euw2-samba-server-c21-01 winbindd[484378]: *[2020/09/30 >>> > 12:58:25.161629, 0] >>> > ../../source3/winbindd/winbindd.c:1462(winbindd_register_handlers)* >>> > >>> > Sep 30 12:58:25 euw2-samba-server-c21-01 winbindd[484378]: * unable to >>> > initialize domain list* >>> >>> This looks to be a generic winbind and domain joining issue, which >>> probably doesn't have anything to do with CTDB. Phew... :-) >>> >>> Searching for "Could not fetch our SID - did we join?" gets a bunch of >>> hits, including this one: >>> >>> >>> http://samba.2283325.n4.nabble.com/Winbind-error-quot-Could-not-fetch-our-SID-did-we-join-quot-td4726277.html >>> >>> Did you use "net ads join" to join the domain? What does "net ads >>> testjoin" say? >>> >>> peace & happiness, >>> martin >>> >>> >> >> -- >> >> BOB BUCK >> SENIOR PLATFORM SOFTWARE ENGINEER >> >> SKIDMORE, OWINGS & MERRILL >> 7 WORLD TRADE CENTER >> 250 GREENWICH STREET >> NEW YORK, NY 10007 >> T (212) 298-9624 >> ROBERT.BUCK at SOM.COM >> > > > -- > > BOB BUCK > SENIOR PLATFORM SOFTWARE ENGINEER > > SKIDMORE, OWINGS & MERRILL > 7 WORLD TRADE CENTER > 250 GREENWICH STREET > NEW YORK, NY 10007 > T (212) 298-9624 > ROBERT.BUCK at SOM.COM >-- BOB BUCK SENIOR PLATFORM SOFTWARE ENGINEER SKIDMORE, OWINGS & MERRILL 7 WORLD TRADE CENTER 250 GREENWICH STREET NEW YORK, NY 10007 T (212) 298-9624 ROBERT.BUCK at SOM.COM
Howdy! Am 10/5/20 um 3:31 PM schrieb Robert Buck via samba:> It seems as though, when I go from `clustering = no` to `clustering = yes`, > if I do a domain join, it will fail.this is however the way how it you're supposed to setup the whole thing. If it's a cluster, set clustering=yes. Make sure you're only starting ctdb and and not smbd and winbindd before joining. Otoh, you can ignore failures from smbd and winbindd if they fail to start, just do the join and then restart both. Cheerio! -slow -- Ralph Boehme, Samba Team https://samba.org/ Samba Developer, SerNet GmbH https://sernet.de/en/samba/ GPG-Fingerprint FAE2C6088A24252051C559E4AA1E9B7126399E46 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/samba/attachments/20201005/61db53bc/signature.sig>
On 05/10/2020 14:31, Robert Buck via samba wrote:> Hi Martin > > It seems as though, when I go from `clustering = no` to `clustering = yes`, > if I do a domain join, it will fail. However, if I do a `systemctl restart > ctdb` (knowing full well it will fail every time), if after this I add a > sleep(15), then do a domain join, then do a `systemctl restart ctdb`, then > the join will have worked, AND CTDB will start properly. So in a nutshell, > in Ansible, >I asked once, but I will ask again, can you please post the smb.conf you are using. Rowland
Hi Bob, On Mon, 5 Oct 2020 09:31:59 -0400, Robert Buck <robert.buck at som.com> wrote:> It seems as though, when I go from `clustering = no` to `clustering = yes`, > if I do a domain join, it will fail. However, if I do a `systemctl restart > ctdb` (knowing full well it will fail every time), if after this I add a > sleep(15), then do a domain join, then do a `systemctl restart ctdb`, then > the join will have worked, AND CTDB will start properly. So in a nutshell, > in Ansible,> - do all the samba setup without clustering on, even winbind setup; verify > it works > - do all the ctdb setup and turn clustering on, but we must again > domain-join, but only after having run restart-ctdb once first, then after > the join, do another restart-ctdb> Only then does the system come to a stable point. > > This appears to be the only way to have a repeatable deployment process of > CTDB over multiple regions globally. > > Any thoughts or recommendations?I think we need to document this better. ;-) Although we've tried to explain things well in the wiki there are still gaps... and this is one of them. Although some of the tutorials around the place are dated they fill in some of these gaps nicely. So, I'll repeat what Ralph said but with a few more words of explanation... :-) When clustering is enabled a new set of databases, managed by CTDB, replaces those that were being used before. This means that even if a node was previously joined to a domain it will no longer be joined after you enable clustering. The credentials have basically disappeared... unless you (immediately?) disable clustering again. In general, before you enable the 49.winbind and 50.samba event scripts, you should start CTDB and join the domain. Then you can enable those scripts and restart CTDB so it will start the services. Since you mention Ansible, I'll point you at autocluster, which I rewrote (last year?) using Vagrant and Ansible. It is a testing tool to generate virtual clusters for (developer) testing of Clustered Samba. It has a lot of clues that need to make their way into documentation. We don't do releases but there is a git repository at: https://git.samba.org/?p=autocluster.git;a=summary Here's the sequence of tasks that we use to configure a "nas" node: https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/main.yml;h=0c444bd77c0a883b1c608fcd6398592be8e962de;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 In particular, this file disables the event scripts: https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/generic/ctdb.yml;h=0271d2a11cff0e9359e115f20c5e641e3279c3ea;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 and later the domain is joined: https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/generic/ctdb-with-samba-nfs.yml;h=b6f9c6d2354e4922535d9048648df4e9e5161689;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 Note that I'm not an Ansible expert and these Ansible playbooks aren't necessarily idempotent. At the moment it all works well enough and I hope to get opportunities to clean it up more later. It is very much aimed at developer testing... but it would be cool if a subset of it could be used to configure "real" Samba clusters. However, given that you mentioned Ansible I figure that it might document certain things for you nice and clearly. It isn't missing anything obvious because we use it to build several test clusters each night. One day later this week I'll try to take a look at the wiki and add some documentation for joining a domain... peace & happiness, martin
Superb. I'll take a look. Thank you On Tue, Oct 6, 2020 at 1:46 AM Martin Schwenke <martin at meltin.net> wrote:> Hi Bob, > > On Mon, 5 Oct 2020 09:31:59 -0400, Robert Buck <robert.buck at som.com> > wrote: > > > It seems as though, when I go from `clustering = no` to `clustering > yes`, > > if I do a domain join, it will fail. However, if I do a `systemctl > restart > > ctdb` (knowing full well it will fail every time), if after this I add a > > sleep(15), then do a domain join, then do a `systemctl restart ctdb`, > then > > the join will have worked, AND CTDB will start properly. So in a > nutshell, > > in Ansible, > > > - do all the samba setup without clustering on, even winbind setup; > verify > > it works > > - do all the ctdb setup and turn clustering on, but we must again > > domain-join, but only after having run restart-ctdb once first, then > after > > the join, do another restart-ctdb > > > Only then does the system come to a stable point. > > > > This appears to be the only way to have a repeatable deployment process > of > > CTDB over multiple regions globally. > > > > Any thoughts or recommendations? > > I think we need to document this better. ;-) > > Although we've tried to explain things well in the wiki there are still > gaps... and this is one of them. Although some of the tutorials around > the place are dated they fill in some of these gaps nicely. > > So, I'll repeat what Ralph said but with a few more words of > explanation... :-) > > When clustering is enabled a new set of databases, managed by CTDB, > replaces those that were being used before. This means that even if a > node was previously joined to a domain it will no longer be joined > after you enable clustering. The credentials have basically > disappeared... unless you (immediately?) disable clustering again. > > In general, before you enable the 49.winbind and 50.samba event > scripts, you should start CTDB and join the domain. > > Then you can enable those scripts and restart CTDB so it will start the > services. > > Since you mention Ansible, I'll point you at autocluster, which I > rewrote (last year?) using Vagrant and Ansible. It is a testing tool > to generate virtual clusters for (developer) testing of Clustered > Samba. It has a lot of clues that need to make their way into > documentation. We don't do releases but there is a git repository at: > > https://git.samba.org/?p=autocluster.git;a=summary > > Here's the sequence of tasks that we use to configure a "nas" node: > > > https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/main.yml;h=0c444bd77c0a883b1c608fcd6398592be8e962de;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 > > In particular, this file disables the event scripts: > > > https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/generic/ctdb.yml;h=0271d2a11cff0e9359e115f20c5e641e3279c3ea;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 > > and later the domain is joined: > > > https://git.samba.org/?p=autocluster.git;a=blob;f=ansible/node/roles/nas/tasks/generic/ctdb-with-samba-nfs.yml;h=b6f9c6d2354e4922535d9048648df4e9e5161689;hb=73b6a2844e827b4c2c2b5d5946cc14c7c61d7d75 > > Note that I'm not an Ansible expert and these Ansible playbooks aren't > necessarily idempotent. At the moment it all works well enough and I > hope to get opportunities to clean it up more later. It is very much > aimed at developer testing... but it would be cool if a subset of it > could be used to configure "real" Samba clusters. > > However, given that you mentioned Ansible I figure that it might > document certain things for you nice and clearly. It isn't missing > anything obvious because we use it to build several test clusters each > night. > > One day later this week I'll try to take a look at the wiki and add some > documentation for joining a domain... > > peace & happiness, > martin > > --BOB BUCK SENIOR PLATFORM SOFTWARE ENGINEER SKIDMORE, OWINGS & MERRILL 7 WORLD TRADE CENTER 250 GREENWICH STREET NEW YORK, NY 10007 T (212) 298-9624 ROBERT.BUCK at SOM.COM