On Thu, 2014-12-04 at 16:46 -0800, Gordon Messmer wrote:> On 12/04/2014 05:45 AM, David McGuffey wrote:> In practice, however, there's a bunch of information you didn't provide, > so some of those steps are wrong. > > I'm not sure what dm-0, dm-2 and dm-3 are, but they're indicated in your > mdstat. I'm guessing that you made partitions, and then made LVM or > crypto devices, and then did RAID on top of that. If either of those > are correct, that's completely the wrong way to build RAID sets. You > risk either bad performance from doing crypto more often than is > required, or possibly corruption as a result of LVM not mapping blocks > the way you expect. > > If you build software RAID, I really strongly recommend that you keep it > as simple as possible. That means a) build sofware RAID sets from raw > partitions and b) use as few partitions as possible. >Gordon, Agree, I've probably made it too complicated. It is a workstation with sensitive data on it so I've encrypted the partitions. md1 is fairly simple...two large disks in raid1, encrypted, and mounted as /home. md0 is probably way too complicated and not a good way to go. The sensitive data in md0 is in /var (virtual machines). I've backed up both /home and /var/lib/libvirt/images, so I think I'll start over on md0 with a new disk and a fresh install. Dave
On Mon, 2014-12-08 at 21:11 -0500, David McGuffey wrote:> On Thu, 2014-12-04 at 16:46 -0800, Gordon Messmer wrote: > > On 12/04/2014 05:45 AM, David McGuffey wrote: > > > In practice, however, there's a bunch of information you didn't provide, > > so some of those steps are wrong. > > > > I'm not sure what dm-0, dm-2 and dm-3 are, but they're indicated in your > > mdstat. I'm guessing that you made partitions, and then made LVM or > > crypto devices, and then did RAID on top of that. If either of those > > are correct, that's completely the wrong way to build RAID sets. You > > risk either bad performance from doing crypto more often than is > > required, or possibly corruption as a result of LVM not mapping blocks > > the way you expect. > > > > If you build software RAID, I really strongly recommend that you keep it > > as simple as possible. That means a) build sofware RAID sets from raw > > partitions and b) use as few partitions as possible. > > > > Gordon, > > Agree, I've probably made it too complicated. It is a workstation with > sensitive data on it so I've encrypted the partitions. > > md1 is fairly simple...two large disks in raid1, encrypted, and mounted > as /home. > > md0 is probably way too complicated and not a good way to go. The > sensitive data in md0 is in /var (virtual machines). > > I've backed up both /home and /var/lib/libvirt/images, so I think I'll > start over on md0 with a new disk and a fresh install. > > Dave >Armed with a backup I decided to use the disk utility GUI to check the array and then re-attach the disk. After a rebuild phase it reattached and the state changed to 'clean.' I rebooted to see if it would stay attached; it did. I'll still get ready for another failure. Will read up on the best methods to have an encrypted filesystem on top of raid-1. Dave M
On 12/08/2014 08:35 PM, David McGuffey wrote:> I'll still get ready for another failure. Will read up on the best > methods to have an encrypted filesystem on top of raid-1.I'm pretty sure that if you tell the Fedora installer to build an encrypted RAID1 system, you'll get exactly what I described previously. In detail: sda1 - 512MB sda2 - remainder of disk sdb1 - 512MB sdb2 - remainder of disk md0 - RAID1 including sda1 and sdb1 md1 - RAID1 including sda2 and sdb2 /boot - filesystem on md0 luks-$(uuid) - encrypted block device on md1 pv.01 - LVM2 physical volume on luks-$(uuid) fedora_$(hostname) - LVM2 volume group including "pv.01" swap - swap on logical volume root - filesystem on logical volume home - filesystem on logical volume If you replace a disk, you'll need to partition it correctly and "mdadm --add" it to the two RAID volumes. If you manually set up some other layering, replacing a disk will probably be more involved.