Thank you for your assistance. Turns out everything is really really screwed up and I can't even add a DC right now. I thought I had added a third and it seemed to be replicating but now the other two won't recognize it as a DC. I'm so deep in this hole right now I'm considering nuking the whole thing, quitting my job and moving to a desert island. It seriously couldn't make it any worse. On Sun, Aug 16, 2020 at 11:39 PM Rowland penny via samba < samba at lists.samba.org> wrote:> On 17/08/2020 04:21, Peter Pollock via samba wrote: > > So I screwed up my AD. > > > > At some point a Windows 2019 server was attempted to be added and it > messed > > the schema up. > > > > That server was removed and ceremonially burned in the fire pit but I > > couldn't fix the issues that had arisen, so I restored both of the > original > > Samba DC's from backups > > That is probably where your main problems start, you should never > restore an individual DC, you only restore a domain. You restored two > DC's from backups, were the backups taken at exactly the same time ? > > What you should have done was, restore one DC, seize all FSMO roles to > this DC, forcibly demote any other DC's and then clean up the other DC's > and rejoined them to the domain as new DC's > > > and everything LOOKS OK on the surface, but they're > > not replicating and for some reason some of the permissions on the drive > > containing the home folders got changed even though the restore had > nothing > > to do with that drive... and there are other weird issues, like the > windows > > computers trying to log on randomly say there is no trust relationship > > between them and the domain. > > > > It's all weird and I can't work out how to fix any of it. > > > > I built a new DC (DC3) and that initially got a copy of everything from > one > > of the original DC's but after that they won't replicate to it. It can > > replicate outbound with both of the other DC's but not inbound and DC1 > and > > DC2 won't replicate with each other at all. > > > > We are a small school and have virtually zero budget (latest DC is built > on > > an old desktop, so it can't stay around forever) but I've managed to get > > funding for a new server, which might help. > > > > One of my big problems is that we only have (or had) 2 servers. I > couldn't > > risk not having a backup DC so I built them both as samba DC's - but one > is > > also the fileserver. (I know, fileserves shouldn't be DC's but I didn't > > have much choice). > > > > I don't know how to fix the problems and have decided I probably need to > > rebuild both of the problem servers from scratch, but my understanding is > > that if I do that, I have to demote them then rebuild them with different > > names so the AD doesn't get confused. Is that right? > You can re-use the names, but you must ensure the DC is fully demoted > and cleaned up before the join. > > > > My issue there is the shares on the fileserver. All the windows drive > > mappings on the client machines are to \\dc02\sharename - and they won't > > work any more if I rebuild with a different server name. Is there a way > to > > migrate them across? > > > > My fear is also that there is something screwy with the schema even now > and > > just rebuilding won't fix that so I'm wondering if I should start again > > from scratch with a completely new domain, somehow import all the user > > details and go around manually rejoining all the workstations. > If you have a backup from before the Windows 2019 DC was joined, you > should be able to fix this, but you will lose any changes made to AD > after the backup was made. > > > > I'm looking for advice here. I know I've messed up bad but I have to try > to > > fix it, although that's hard because now school is back in session I > can't > > do anything during the day, partly because I'm teaching and partly > because > > the teachers rely on the network for their teaching. I'm going to take > the > > school offline in a couple of weeks for an entire weekend to fix this > > nonsense... I just don't know what the best way to go about fixing it is? > > It sounds like you now have three machines to use, perhaps use one as a > Unix domain member running as a fileserver and the other two as DC's. > > Rowland > > > > -- > To unsubscribe from this list go to the following URL and read the > instructions: https://lists.samba.org/mailman/options/samba >
Given the situation in the world today, I would agree a desert island sounds ideal. However I think there is hope for your network, so you can run away in peace. First, you have not said which Samba versions you are using, so the advise below is a bit generic. If my network was behaving like this I would use Samba's backup tools to make a backup, and start again with the restored domain, again using our restore tools. The restore will manage the FSMO roles and clean up pile of stuff as part of removing the other domain controllers. samba-tool domain backup samba-tool domain restore There are offline and online backup options, I would try both and see which works given the situations you are in. Also, the schema should be fixable. We have tools to load a new schema into Samba - you might be missing the schema file source4/setup/adprep/samba-4.7-missing-for-schema45.ldif This is in the tarball (but not installed, a bug) and consists of the difference between the schema we had and the schema we claimed to have as schema version 45. In terms of the permissions, you may have lost idmap.ldb, so the mapping between unix uids and Samba SIDs would be scrambled. Do note that this is not replicated either. I hope these suggestions help. Andrew Bartlett On Sat, 2020-08-29 at 14:44 -0700, Peter Pollock via samba wrote:> Thank you for your assistance. > > Turns out everything is really really screwed up and I can't even add > a DC > right now. I thought I had added a third and it seemed to be > replicating > but now the other two won't recognize it as a DC. > > I'm so deep in this hole right now I'm considering nuking the whole > thing, > quitting my job and moving to a desert island. It seriously couldn't > make > it any worse. > > On Sun, Aug 16, 2020 at 11:39 PM Rowland penny via samba < > samba at lists.samba.org> wrote: > > > On 17/08/2020 04:21, Peter Pollock via samba wrote: > > > So I screwed up my AD. > > > > > > At some point a Windows 2019 server was attempted to be added and > > > it > > > > messed > > > the schema up. > > > > > > That server was removed and ceremonially burned in the fire pit > > > but I > > > couldn't fix the issues that had arisen, so I restored both of > > > the > > > > original > > > Samba DC's from backups > > > > That is probably where your main problems start, you should never > > restore an individual DC, you only restore a domain. You restored > > two > > DC's from backups, were the backups taken at exactly the same time > > ? > > > > What you should have done was, restore one DC, seize all FSMO roles > > to > > this DC, forcibly demote any other DC's and then clean up the other > > DC's > > and rejoined them to the domain as new DC's > > > > > and everything LOOKS OK on the surface, but they're > > > not replicating and for some reason some of the permissions on > > > the drive > > > containing the home folders got changed even though the restore > > > had > > > > nothing > > > to do with that drive... and there are other weird issues, like > > > the > > > > windows > > > computers trying to log on randomly say there is no trust > > > relationship > > > between them and the domain. > > > > > > It's all weird and I can't work out how to fix any of it. > > > > > > I built a new DC (DC3) and that initially got a copy of > > > everything from > > > > one > > > of the original DC's but after that they won't replicate to it. > > > It can > > > replicate outbound with both of the other DC's but not inbound > > > and DC1 > > > > and > > > DC2 won't replicate with each other at all. > > > > > > We are a small school and have virtually zero budget (latest DC > > > is built > > > > on > > > an old desktop, so it can't stay around forever) but I've managed > > > to get > > > funding for a new server, which might help. > > > > > > One of my big problems is that we only have (or had) 2 servers. I > > > > couldn't > > > risk not having a backup DC so I built them both as samba DC's - > > > but one > > > > is > > > also the fileserver. (I know, fileserves shouldn't be DC's but I > > > didn't > > > have much choice). > > > > > > I don't know how to fix the problems and have decided I probably > > > need to > > > rebuild both of the problem servers from scratch, but my > > > understanding is > > > that if I do that, I have to demote them then rebuild them with > > > different > > > names so the AD doesn't get confused. Is that right? > > > > You can re-use the names, but you must ensure the DC is fully > > demoted > > and cleaned up before the join. > > > > > > My issue there is the shares on the fileserver. All the windows > > > drive > > > mappings on the client machines are to \\dc02\sharename - and > > > they won't > > > work any more if I rebuild with a different server name. Is there > > > a way > > > > to > > > migrate them across? > > > > > > My fear is also that there is something screwy with the schema > > > even now > > > > and > > > just rebuilding won't fix that so I'm wondering if I should start > > > again > > > from scratch with a completely new domain, somehow import all the > > > user > > > details and go around manually rejoining all the workstations. > > > > If you have a backup from before the Windows 2019 DC was joined, > > you > > should be able to fix this, but you will lose any changes made to > > AD > > after the backup was made. > > > > > > I'm looking for advice here. I know I've messed up bad but I have > > > to try > > > > to > > > fix it, although that's hard because now school is back in > > > session I > > > > can't > > > do anything during the day, partly because I'm teaching and > > > partly > > > > because > > > the teachers rely on the network for their teaching. I'm going to > > > take > > > > the > > > school offline in a couple of weeks for an entire weekend to fix > > > this > > > nonsense... I just don't know what the best way to go about > > > fixing it is? > > > > It sounds like you now have three machines to use, perhaps use one > > as a > > Unix domain member running as a fileserver and the other two as > > DC's. > > > > Rowland > > > > > > > > -- > > To unsubscribe from this list go to the following URL and read the > > instructions: https://lists.samba.org/mailman/options/samba > >-- Andrew Bartlett https://samba.org/~abartlet/ Authentication Developer, Samba Team https://samba.org Samba Developer, Catalyst IT https://catalyst.net.nz/services/samba
Of course, it seems we are running Samba 4.7.6-Ubuntu. Great. The biggest regret of my last few years is that we are also running Zentyal, which I am desperate to get rid of. I'm trying to build a new server with the latest Samba and join that to the domain to see if I can get a good copy of the database, then wipe the other servers and rebuild them from scratch On Sat, Aug 29, 2020 at 3:04 PM Andrew Bartlett <abartlet at samba.org> wrote:> Given the situation in the world today, I would agree a desert island > sounds ideal. > > However I think there is hope for your network, so you can run away in > peace. > > First, you have not said which Samba versions you are using, so the > advise below is a bit generic. > > If my network was behaving like this I would use Samba's backup tools > to make a backup, and start again with the restored domain, again using > our restore tools. The restore will manage the FSMO roles and clean up > pile of stuff as part of removing the other domain controllers. > > samba-tool domain backup > samba-tool domain restore > > There are offline and online backup options, I would try both and see > which works given the situations you are in. > > Also, the schema should be fixable. We have tools to load a new schema > into Samba - you might be missing the schema file > > source4/setup/adprep/samba-4.7-missing-for-schema45.ldif > > This is in the tarball (but not installed, a bug) and consists of the > difference between the schema we had and the schema we claimed to have > as schema version 45. > > In terms of the permissions, you may have lost idmap.ldb, so the > mapping between unix uids and Samba SIDs would be scrambled. Do note > that this is not replicated either. > > I hope these suggestions help. > > Andrew Bartlett > > On Sat, 2020-08-29 at 14:44 -0700, Peter Pollock via samba wrote: > > Thank you for your assistance. > > > > Turns out everything is really really screwed up and I can't even add > > a DC > > right now. I thought I had added a third and it seemed to be > > replicating > > but now the other two won't recognize it as a DC. > > > > I'm so deep in this hole right now I'm considering nuking the whole > > thing, > > quitting my job and moving to a desert island. It seriously couldn't > > make > > it any worse. > > > > On Sun, Aug 16, 2020 at 11:39 PM Rowland penny via samba < > > samba at lists.samba.org> wrote: > > > > > On 17/08/2020 04:21, Peter Pollock via samba wrote: > > > > So I screwed up my AD. > > > > > > > > At some point a Windows 2019 server was attempted to be added and > > > > it > > > > > > messed > > > > the schema up. > > > > > > > > That server was removed and ceremonially burned in the fire pit > > > > but I > > > > couldn't fix the issues that had arisen, so I restored both of > > > > the > > > > > > original > > > > Samba DC's from backups > > > > > > That is probably where your main problems start, you should never > > > restore an individual DC, you only restore a domain. You restored > > > two > > > DC's from backups, were the backups taken at exactly the same time > > > ? > > > > > > What you should have done was, restore one DC, seize all FSMO roles > > > to > > > this DC, forcibly demote any other DC's and then clean up the other > > > DC's > > > and rejoined them to the domain as new DC's > > > > > > > and everything LOOKS OK on the surface, but they're > > > > not replicating and for some reason some of the permissions on > > > > the drive > > > > containing the home folders got changed even though the restore > > > > had > > > > > > nothing > > > > to do with that drive... and there are other weird issues, like > > > > the > > > > > > windows > > > > computers trying to log on randomly say there is no trust > > > > relationship > > > > between them and the domain. > > > > > > > > It's all weird and I can't work out how to fix any of it. > > > > > > > > I built a new DC (DC3) and that initially got a copy of > > > > everything from > > > > > > one > > > > of the original DC's but after that they won't replicate to it. > > > > It can > > > > replicate outbound with both of the other DC's but not inbound > > > > and DC1 > > > > > > and > > > > DC2 won't replicate with each other at all. > > > > > > > > We are a small school and have virtually zero budget (latest DC > > > > is built > > > > > > on > > > > an old desktop, so it can't stay around forever) but I've managed > > > > to get > > > > funding for a new server, which might help. > > > > > > > > One of my big problems is that we only have (or had) 2 servers. I > > > > > > couldn't > > > > risk not having a backup DC so I built them both as samba DC's - > > > > but one > > > > > > is > > > > also the fileserver. (I know, fileserves shouldn't be DC's but I > > > > didn't > > > > have much choice). > > > > > > > > I don't know how to fix the problems and have decided I probably > > > > need to > > > > rebuild both of the problem servers from scratch, but my > > > > understanding is > > > > that if I do that, I have to demote them then rebuild them with > > > > different > > > > names so the AD doesn't get confused. Is that right? > > > > > > You can re-use the names, but you must ensure the DC is fully > > > demoted > > > and cleaned up before the join. > > > > > > > > My issue there is the shares on the fileserver. All the windows > > > > drive > > > > mappings on the client machines are to \\dc02\sharename - and > > > > they won't > > > > work any more if I rebuild with a different server name. Is there > > > > a way > > > > > > to > > > > migrate them across? > > > > > > > > My fear is also that there is something screwy with the schema > > > > even now > > > > > > and > > > > just rebuilding won't fix that so I'm wondering if I should start > > > > again > > > > from scratch with a completely new domain, somehow import all the > > > > user > > > > details and go around manually rejoining all the workstations. > > > > > > If you have a backup from before the Windows 2019 DC was joined, > > > you > > > should be able to fix this, but you will lose any changes made to > > > AD > > > after the backup was made. > > > > > > > > I'm looking for advice here. I know I've messed up bad but I have > > > > to try > > > > > > to > > > > fix it, although that's hard because now school is back in > > > > session I > > > > > > can't > > > > do anything during the day, partly because I'm teaching and > > > > partly > > > > > > because > > > > the teachers rely on the network for their teaching. I'm going to > > > > take > > > > > > the > > > > school offline in a couple of weeks for an entire weekend to fix > > > > this > > > > nonsense... I just don't know what the best way to go about > > > > fixing it is? > > > > > > It sounds like you now have three machines to use, perhaps use one > > > as a > > > Unix domain member running as a fileserver and the other two as > > > DC's. > > > > > > Rowland > > > > > > > > > > > > -- > > > To unsubscribe from this list go to the following URL and read the > > > instructions: https://lists.samba.org/mailman/options/samba > > > > -- > Andrew Bartlett https://samba.org/~abartlet/ > Authentication Developer, Samba Team https://samba.org > Samba Developer, Catalyst IT > https://catalyst.net.nz/services/samba > > > >