So I screwed up my AD. At some point a Windows 2019 server was attempted to be added and it messed the schema up. That server was removed and ceremonially burned in the fire pit but I couldn't fix the issues that had arisen, so I restored both of the original Samba DC's from backups and everything LOOKS OK on the surface, but they're not replicating and for some reason some of the permissions on the drive containing the home folders got changed even though the restore had nothing to do with that drive... and there are other weird issues, like the windows computers trying to log on randomly say there is no trust relationship between them and the domain. It's all weird and I can't work out how to fix any of it. I built a new DC (DC3) and that initially got a copy of everything from one of the original DC's but after that they won't replicate to it. It can replicate outbound with both of the other DC's but not inbound and DC1 and DC2 won't replicate with each other at all. We are a small school and have virtually zero budget (latest DC is built on an old desktop, so it can't stay around forever) but I've managed to get funding for a new server, which might help. One of my big problems is that we only have (or had) 2 servers. I couldn't risk not having a backup DC so I built them both as samba DC's - but one is also the fileserver. (I know, fileserves shouldn't be DC's but I didn't have much choice). I don't know how to fix the problems and have decided I probably need to rebuild both of the problem servers from scratch, but my understanding is that if I do that, I have to demote them then rebuild them with different names so the AD doesn't get confused. Is that right? My issue there is the shares on the fileserver. All the windows drive mappings on the client machines are to \\dc02\sharename - and they won't work any more if I rebuild with a different server name. Is there a way to migrate them across? My fear is also that there is something screwy with the schema even now and just rebuilding won't fix that so I'm wondering if I should start again from scratch with a completely new domain, somehow import all the user details and go around manually rejoining all the workstations. I'm looking for advice here. I know I've messed up bad but I have to try to fix it, although that's hard because now school is back in session I can't do anything during the day, partly because I'm teaching and partly because the teachers rely on the network for their teaching. I'm going to take the school offline in a couple of weeks for an entire weekend to fix this nonsense... I just don't know what the best way to go about fixing it is?
My first suggestion is to make a list. Let's start here: 1. Backup all user data to an external drive or a backup server that no one else can touch. You'll need this later. Choose a backup method that allows snapshots or versioning, so that you can do incremental backups. This step can likely run at night when usage is low. 2. Choose a date in the (close) future, and give your staff warning that the network will be going down, save their work by 3:00pm, etc. At this point, *stop access from all other network sources, except for your SSH.* This way, you can create a good backup, and the staff know to expect it. 3. Get a moderately new computer that supports virtualization (most likely, anything from the last 10 years or so would support it). Install a hypervisor to it (if you're using Linux, you can use Xen or KVM; if you need something with point-and-click, check out ProxMox). Create a VM, called "DC-50" (guaranteed that's not a name you have in use ;-) ) 4. Get that DC promoted, and transfer all the FSMO roles to it. If roles won't go willingly, then force it (seize the roles). The Samba Wiki has great documentation on this. 5. Once you are sure the new DC is running, shut down the first DC, leaving this new one running. Can computers log in still? New users exist? Replication with DC3 works? Great! 6. Decommission the first DC, then repeat with DC2. At this point, you should only have DC3 and DC-50 running as domain controllers, and no file servers. *This is OK* 7. Create a new VM to act as a file server (if you have the space). If not, setup your new server to be the file server, and join it *as a member server* to the domain. Copy all the user data back to this file server. Setup permissions if they seem wonky (you can script this if you know your bash commands). 8. Learn to use Group Policy Objects (GPOs) to map network drives, based on the user group or user name. This way, you have one central location to change a shared folder or server name, going forward. 9. ??? 10. PROFIT On Sun, Aug 16, 2020 at 8:23 PM Peter Pollock via samba < samba at lists.samba.org> wrote:> So I screwed up my AD. > > At some point a Windows 2019 server was attempted to be added and it messed > the schema up. > > That server was removed and ceremonially burned in the fire pit but I > couldn't fix the issues that had arisen, so I restored both of the original > Samba DC's from backups and everything LOOKS OK on the surface, but they're > not replicating and for some reason some of the permissions on the drive > containing the home folders got changed even though the restore had nothing > to do with that drive... and there are other weird issues, like the windows > computers trying to log on randomly say there is no trust relationship > between them and the domain. > > It's all weird and I can't work out how to fix any of it. > > I built a new DC (DC3) and that initially got a copy of everything from one > of the original DC's but after that they won't replicate to it. It can > replicate outbound with both of the other DC's but not inbound and DC1 and > DC2 won't replicate with each other at all. > > We are a small school and have virtually zero budget (latest DC is built on > an old desktop, so it can't stay around forever) but I've managed to get > funding for a new server, which might help. > > One of my big problems is that we only have (or had) 2 servers. I couldn't > risk not having a backup DC so I built them both as samba DC's - but one is > also the fileserver. (I know, fileserves shouldn't be DC's but I didn't > have much choice). > > I don't know how to fix the problems and have decided I probably need to > rebuild both of the problem servers from scratch, but my understanding is > that if I do that, I have to demote them then rebuild them with different > names so the AD doesn't get confused. Is that right? > > My issue there is the shares on the fileserver. All the windows drive > mappings on the client machines are to \\dc02\sharename - and they won't > work any more if I rebuild with a different server name. Is there a way to > migrate them across? > > My fear is also that there is something screwy with the schema even now and > just rebuilding won't fix that so I'm wondering if I should start again > from scratch with a completely new domain, somehow import all the user > details and go around manually rejoining all the workstations. > > I'm looking for advice here. I know I've messed up bad but I have to try to > fix it, although that's hard because now school is back in session I can't > do anything during the day, partly because I'm teaching and partly because > the teachers rely on the network for their teaching. I'm going to take the > school offline in a couple of weeks for an entire weekend to fix this > nonsense... I just don't know what the best way to go about fixing it is? > -- > To unsubscribe from this list go to the following URL and read the > instructions: https://lists.samba.org/mailman/options/samba >
On 17/08/2020 04:21, Peter Pollock via samba wrote:> So I screwed up my AD. > > At some point a Windows 2019 server was attempted to be added and it messed > the schema up. > > That server was removed and ceremonially burned in the fire pit but I > couldn't fix the issues that had arisen, so I restored both of the original > Samba DC's from backupsThat is probably where your main problems start, you should never restore an individual DC, you only restore a domain. You restored two DC's from backups, were the backups taken at exactly the same time ? What you should have done was, restore one DC, seize all FSMO roles to this DC, forcibly demote any other DC's and then clean up the other DC's and rejoined them to the domain as new DC's> and everything LOOKS OK on the surface, but they're > not replicating and for some reason some of the permissions on the drive > containing the home folders got changed even though the restore had nothing > to do with that drive... and there are other weird issues, like the windows > computers trying to log on randomly say there is no trust relationship > between them and the domain. > > It's all weird and I can't work out how to fix any of it. > > I built a new DC (DC3) and that initially got a copy of everything from one > of the original DC's but after that they won't replicate to it. It can > replicate outbound with both of the other DC's but not inbound and DC1 and > DC2 won't replicate with each other at all. > > We are a small school and have virtually zero budget (latest DC is built on > an old desktop, so it can't stay around forever) but I've managed to get > funding for a new server, which might help. > > One of my big problems is that we only have (or had) 2 servers. I couldn't > risk not having a backup DC so I built them both as samba DC's - but one is > also the fileserver. (I know, fileserves shouldn't be DC's but I didn't > have much choice). > > I don't know how to fix the problems and have decided I probably need to > rebuild both of the problem servers from scratch, but my understanding is > that if I do that, I have to demote them then rebuild them with different > names so the AD doesn't get confused. Is that right?You can re-use the names, but you must ensure the DC is fully demoted and cleaned up before the join.> > My issue there is the shares on the fileserver. All the windows drive > mappings on the client machines are to \\dc02\sharename - and they won't > work any more if I rebuild with a different server name. Is there a way to > migrate them across? > > My fear is also that there is something screwy with the schema even now and > just rebuilding won't fix that so I'm wondering if I should start again > from scratch with a completely new domain, somehow import all the user > details and go around manually rejoining all the workstations.If you have a backup from before the Windows 2019 DC was joined, you should be able to fix this, but you will lose any changes made to AD after the backup was made.> > I'm looking for advice here. I know I've messed up bad but I have to try to > fix it, although that's hard because now school is back in session I can't > do anything during the day, partly because I'm teaching and partly because > the teachers rely on the network for their teaching. I'm going to take the > school offline in a couple of weeks for an entire weekend to fix this > nonsense... I just don't know what the best way to go about fixing it is?It sounds like you now have three machines to use, perhaps use one as a Unix domain member running as a fileserver and the other two as DC's. Rowland
Thank you for your assistance. Turns out everything is really really screwed up and I can't even add a DC right now. I thought I had added a third and it seemed to be replicating but now the other two won't recognize it as a DC. I'm so deep in this hole right now I'm considering nuking the whole thing, quitting my job and moving to a desert island. It seriously couldn't make it any worse. On Sun, Aug 16, 2020 at 11:39 PM Rowland penny via samba < samba at lists.samba.org> wrote:> On 17/08/2020 04:21, Peter Pollock via samba wrote: > > So I screwed up my AD. > > > > At some point a Windows 2019 server was attempted to be added and it > messed > > the schema up. > > > > That server was removed and ceremonially burned in the fire pit but I > > couldn't fix the issues that had arisen, so I restored both of the > original > > Samba DC's from backups > > That is probably where your main problems start, you should never > restore an individual DC, you only restore a domain. You restored two > DC's from backups, were the backups taken at exactly the same time ? > > What you should have done was, restore one DC, seize all FSMO roles to > this DC, forcibly demote any other DC's and then clean up the other DC's > and rejoined them to the domain as new DC's > > > and everything LOOKS OK on the surface, but they're > > not replicating and for some reason some of the permissions on the drive > > containing the home folders got changed even though the restore had > nothing > > to do with that drive... and there are other weird issues, like the > windows > > computers trying to log on randomly say there is no trust relationship > > between them and the domain. > > > > It's all weird and I can't work out how to fix any of it. > > > > I built a new DC (DC3) and that initially got a copy of everything from > one > > of the original DC's but after that they won't replicate to it. It can > > replicate outbound with both of the other DC's but not inbound and DC1 > and > > DC2 won't replicate with each other at all. > > > > We are a small school and have virtually zero budget (latest DC is built > on > > an old desktop, so it can't stay around forever) but I've managed to get > > funding for a new server, which might help. > > > > One of my big problems is that we only have (or had) 2 servers. I > couldn't > > risk not having a backup DC so I built them both as samba DC's - but one > is > > also the fileserver. (I know, fileserves shouldn't be DC's but I didn't > > have much choice). > > > > I don't know how to fix the problems and have decided I probably need to > > rebuild both of the problem servers from scratch, but my understanding is > > that if I do that, I have to demote them then rebuild them with different > > names so the AD doesn't get confused. Is that right? > You can re-use the names, but you must ensure the DC is fully demoted > and cleaned up before the join. > > > > My issue there is the shares on the fileserver. All the windows drive > > mappings on the client machines are to \\dc02\sharename - and they won't > > work any more if I rebuild with a different server name. Is there a way > to > > migrate them across? > > > > My fear is also that there is something screwy with the schema even now > and > > just rebuilding won't fix that so I'm wondering if I should start again > > from scratch with a completely new domain, somehow import all the user > > details and go around manually rejoining all the workstations. > If you have a backup from before the Windows 2019 DC was joined, you > should be able to fix this, but you will lose any changes made to AD > after the backup was made. > > > > I'm looking for advice here. I know I've messed up bad but I have to try > to > > fix it, although that's hard because now school is back in session I > can't > > do anything during the day, partly because I'm teaching and partly > because > > the teachers rely on the network for their teaching. I'm going to take > the > > school offline in a couple of weeks for an entire weekend to fix this > > nonsense... I just don't know what the best way to go about fixing it is? > > It sounds like you now have three machines to use, perhaps use one as a > Unix domain member running as a fileserver and the other two as DC's. > > Rowland > > > > -- > To unsubscribe from this list go to the following URL and read the > instructions: https://lists.samba.org/mailman/options/samba >