Hi, maybe this thread should ask an even more general question: How are AD-DC admins supposed to test upgrades? If there is more than one AD-DC, an upgraded DC which causes problems cannot (must not) be restored from backup. This is one of the reasons why I do not want to switch to AD. A PDC *can* simply be restored from backup. It is even enough to switch back to the previous contents of /usr/local/samba, a matter of seconds. Should we force all clients to use/test the upgraded DC by switching the other DCs off, and in case of problems turn the new DC off, the others back on, and then wipe the upgraded DC? Is it enough to wipe /usr/local/bin, make install, rejoin, copy sysvol? regards, Klaus
On Tue, 19 Apr 2016, Klaus Hartnegg wrote:> How are AD-DC admins supposed to test upgrades? If there is more than one > AD-DC, an upgraded DC which causes problems cannot (must not) be restored > from backup. This is one of the reasons why I do not want to switch to AD. A > PDC *can* simply be restored from backup. It is even enough to switch back to > the previous contents of /usr/local/samba, a matter of seconds.Note that if you only have a single DC, you can restore from backup (or a previous copy of /usr/local/samba). So if you only have a single PDC and no BDCs, you are in the same position. It's only when you have multiple live DCs that it becomes a problem. My method for upgrading has been to upgrade the primary DC that holds all of the FSMO roles first. If there are no problems after a day or two, upgrade the rest. If there are problems, transfer the roles to another DC and rebuild the former primary. If you have your FSMO roles split between DCs, that would be a little more complicated, obviously. I have yet to have any major problems that required a downgrade, though I generally stay on the older stable releases and don't rush to upgrade until all the problems on the mailing list seem to have been sorted out. I have had a few issues with leftover entries in the LDAP database I had to clean up manually, but that's about it. I recall a samba dev on this list recently said newer versions are supposed to be better about not doing that, so there may be some advantages to using newer versions... BTW, I seem to recall reading that either Windows Server 2016 or the next client version of windows will drop support for NT4 domains entirely, so it's only a matter of time before NT4 domains are no longer an option.
On 04/19/2016 07:39 AM, Klaus Hartnegg wrote:> Hi, > > maybe this thread should ask an even more general question: > > How are AD-DC admins supposed to test upgrades? If there is more than > one AD-DC, an upgraded DC which causes problems cannot (must not) be > restored from backup. This is one of the reasons why I do not want to > switch to AD. A PDC *can* simply be restored from backup. It is even > enough to switch back to the previous contents of /usr/local/samba, a > matter of seconds. > > Should we force all clients to use/test the upgraded DC by switching the > other DCs off, and in case of problems turn the new DC off, the others > back on, and then wipe the upgraded DC? Is it enough to wipe > /usr/local/bin, make install, rejoin, copy sysvol? > > regards, > Klaus >I clone my production DCs, and a number of other systems into a test environment. Then test out the changes within that environment. Once everything looks good, I apply the same changes to production. -- John Yocum, Systems Administrator, DEOHS
On 4/19/2016 10:39 AM, Klaus Hartnegg wrote:> Hi, > > maybe this thread should ask an even more general question: > > How are AD-DC admins supposed to test upgrades? If there is more than > one AD-DC, an upgraded DC which causes problems cannot (must not) be > restored from backup. This is one of the reasons why I do not want to > switch to AD. A PDC *can* simply be restored from backup. It is even > enough to switch back to the previous contents of /usr/local/samba, a > matter of seconds. > > Should we force all clients to use/test the upgraded DC by switching > the other DCs off, and in case of problems turn the new DC off, the > others back on, and then wipe the upgraded DC? Is it enough to wipe > /usr/local/bin, make install, rejoin, copy sysvol? > > regards, > KlausI follow the wiki. Haven't had any issues yet. https://wiki.samba.org/index.php/Updating_Samba I also suggest a test environment to test any changes if possible. -- -James
On 04/19/2016 04:39 PM, Klaus Hartnegg wrote:> Hi, > > maybe this thread should ask an even more general question: > > How are AD-DC admins supposed to test upgrades? If there is more than > one AD-DC, an upgraded DC which causes problems cannot (must not) be > restored from backup. This is one of the reasons why I do not want to > switch to AD. A PDC *can* simply be restored from backup. It is even > enough to switch back to the previous contents of /usr/local/samba, a > matter of seconds.Since the upgrade to an AD domain (1.5 years) we have several issues with samba AD and we never achieved the feeling of being in control. To be fair samba has a much bigger responsibility than before and we can do things we couldn't before. We are deploying our software with ansible and dynamically generating docker containers - I'm trying to create an ansible playbook for samba with three different flavors: 1. Create new domain controller from template for desaster recovery: Install configuration, add users, add groups, add dns entries - all from (plain text) backup. 2. Add new domain controller to existing domain (Join, rsync sysvol ..) 3. Create domain member (file server): Install configuration, pam... --> done already This is a lot of effort because we didn't really understand how to continuously backup and replay a running domain. Samba gives us so much headache that it is worth to dedicate some time in this to get it right. These playbooks can be run on vms, containers and real server for continuously testing. Yup we're paranoid. I believe the truth is to have a replay strategy and testing it automatically with a backup, this is at least for me the only confidence I get running software we are depending on. Samba has to be treated like other (web-)software and continuous integration is the key here - then the update day is not always so frightening. Samba is great but my 2 cents it would be helpful if samba development would focus a bit more on integrated backup strategies and easy robust deployment. These things come in mind: - sysvol replication integrated - automatic promoting/joining/replicating etc. I never wan't to be bothered using samba-tool for normal replicated setup - One namespace in config should do the trick. - integrated backup tool for everything that needs to be backupd to bring up the same domain with one command: samba import backup Not to get me wrong a lot of issues are already been working on and a lot of things have been simplified. Samba should feel more like i.e. MongoDB where it is trivial to add and remove instances dynamically, making backups, but of course that is comparing different things. Hope this helps Johannes> > Should we force all clients to use/test the upgraded DC by switching > the other DCs off, and in case of problems turn the new DC off, the > others back on, and then wipe the upgraded DC? Is it enough to wipe > /usr/local/bin, make install, rejoin, copy sysvol? > > regards, > Klaus >-- Johannes Amorosa | Celluloid VFX Celluloid Visual Effects GmbH & Co. KG Paul-Lincke-Ufer 39/40, 10999 Berlin
On Wed, 2016-04-20 at 09:29 +0200, Johannes Amorosa | Celluloid VFX wrote:> > This is a lot of effort because we didn't really understand how to > continuously backup and replay a running domain. Samba gives us so > much > headache that it is > worth to dedicate some time in this to get it right.The difficultly with backups of running domains is that because of the replicated state, restoring an old backup can cause trouble. We suggest instead re-joining, realising that this can also be less than ideal, but seems to work better.> These playbooks can be run on vms, containers and real server for > continuously testing. Yup we're paranoid.> Samba is great but my 2 cents it would be helpful if samba > development > would focus a bit more on integrated backup strategies and easy > robust > deployment. These things come in mind: > > - sysvol replication integratedI agree, this is a major missing part. We had some developers who were interested in implementing this in a protocol-compatible way (ie, able to replicate with Windows), but no progress has been made in a couple of years. Since we released Samba4, I have realised that we have produced a great mimic of AD, but we have lost a little of what has always made Samba great, that it needs to work with windows clients and tools, but not be Windows - we can have some different behaviours. It would be reasonable to have a Samba-only sysvol sync tool, as long as we have a way to import from Windows AD, as most folks don't run mixed Samba -Windows domains after migration.> - automatic promoting/joining/replicating etc. I never wan't to be > bothered using samba-tool for normal replicated setup - One namespace > in > config should do the trick.I understand the frustration here. The pattern comes from how Windows handles it, and the need for a privileged account (the administrator) to create the new DC. It should only be needed once, and in an ideal world, it should start ongoing replication once Samba starts. I take it you would like Samba to be able to re-import the replicated state from just a realm and a DC secret in secrets.ldb? I guess a 'samba-tool domain dcpromo' could detect that we have a password to a working account and just re-sync from there. Would this be helpful?> - integrated backup tool for everything that needs to be backupd to > bring up the same domain with one command: samba import backup > > Not to get me wrong a lot of issues are already been working on and a > lot of things have been simplified. > > Samba should feel more like i.e. MongoDB where it is trivial to add > and > remove instances dynamically, making backups, but of course that is > comparing different things. > Hope this helps > JohannesOne of the big challenges here is that while the Samba Team is large, and we take great pride in supporting all of Samba, the number of us who work daily on AD DC development is much more limited, and our time, particularly for larger tasks, is essentially dictated by the needs of our consulting/development clients. The good news is that pretty much everyone uses Samba4 in much the same way, so the tasks that come up and the patches they produce and the experiences we gain end up helping us all. (I wrote with amusement about this on the other list just earlier today). For example, I think I have, once and for all finally sorted out some really nasty issues in our replication code. Patches for that should be in master soon. The bad news is that to get prompt attention to your particular issue, when it is a larger one, you may need to contract one of the firms employing Samba developers for commercial support: https://www.samba.org/samba/support/globalsupport.html The other part of this is that we would love to have a class of sysadmin-developer folks regularly contributing to Samba. Having most of our team being very heavy into the development, while not often deploying Samba in the real world, means we just don't see the things that truly get under your skin. Writing about your experiences is great, even better would be writing patches. We developed samba-tool in python in the hope that it would be easier to contribute to than our C code. Finally, if you wanted to take on the idea of re-replicating in using just the DC secret, or writing a better backup script (integrated into our make test so we know it just works every time), I would be happy to guide you. Thanks, Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba