I've just finished installing a new Bacula storeage server. Prior to doing the install I did some research and ended up deciding to do the following config. 6x4TB drives /boot/efi efi_fs sda1 /boot/efi_copy efi_fs sdb1 /boot xfs RAID1 sda2 sdb2 VG RAID6 all drives containing SWAP / /home /var/bacula Questions: 1) The big problem with this is that it is dependant on sda for booting. I did find an aritcle on how to set up boot loading on multiple HDD's, including cloning /boot/efi but I now can't find it. Does anyone know of a similar article? 2) is putting SWAP in a RAID a good idea? Will it help, will it cause problems?
> I've just finished installing a new Bacula storeage server. Prior to doing > the > install I did some research and ended up deciding to do the following > config. > > 6x4TB drives > /boot/efi efi_fs sda1 > /boot/efi_copy efi_fs sdb1 > /boot xfs RAID1 sda2 sdb2 > VG RAID6 all drives containing > SWAP > / > /home > /var/bacula > > Questions: > > 1) The big problem with this is that it is dependant on sda for booting. > I > did find an aritcle on how to set up boot loading on multiple HDD's, > including cloning /boot/efi but I now can't find it. Does anyone know of > a > similar article?I also spent (wasted?) quite some time on this issue because I couldn't believe things don't work so nice with EFI as they did before. The designers of EFI obviously forgot that some people might want to boot from software RAID in a redundant way. I ended up with a similar design than you, my fstab has this: /dev/md0 /boot xfs defaults 0 0 /dev/nvme0n1p1 /boot/efi vfat umask=0077,shortname=winnt 0 0 /dev/nvme1n1p1 /boot/efi.backup vfat umask=0077,shortname=winnt 0 0 Then in my package update tool I have a hook which syncs like this: EFISRC="/boot/efi" EFIDEST="${EFISRC}.backup" efisync() { if [ -d "${EFISRC}/EFI" -a -d "${EFIDEST}/EFI" ]; then rsync --archive --delete --verbose "${EFISRC}/EFI" "${EFIDEST}/" fi } BTW, another method could be to put /boot/efi on RAID1 with metadata version 1.0 but that doesn't seem to be reliable, it works for some systems but fails on others according to report I read.> > 2) is putting SWAP in a RAID a good idea? Will it help, will it cause > problems?No problem at all and I don't want to lose a swap device if a disk fails. So it's the correct way to put in on RAID, IMHO. Regards, Simon
On 1/9/19 2:30 AM, Gary Stainburn wrote:> 1) The big problem with this is that it is dependant on sda for booting. I > did find an aritcle on how to set up boot loading on multiple HDD's, > including cloning /boot/efi but I now can't find it. Does anyone know of a > similar article?Use RAID1 for /boot/efi as well.? The installer should get the details right.> 2) is putting SWAP in a RAID a good idea? Will it help, will it cause > problems?It'll be moderately more reliable.? If you have swap on a non-redundant disk and the kernel tries to read it, bad things (TM) will happen. The only "drawback" that I'm aware of is that RAID consistency checks become meaningless, because it's common for swap writes to be canceled before complete, in which case one disk will have the page written but the other won't.? This is by design, and considered the optimal operation.? However, consistency checks don't exclude blocks used for swap, and they'll typically show mismatched blocks.
On 2019-01-09, Gordon Messmer <gordon.messmer at gmail.com> wrote:> On 1/9/19 2:30 AM, Gary Stainburn wrote: > >> 2) is putting SWAP in a RAID a good idea? Will it help, will it cause >> problems? > > The only "drawback" that I'm aware of is that RAID consistency checks > become meaningless, because it's common for swap writes to be canceled > before complete, in which case one disk will have the page written but > the other won't.? This is by design, and considered the optimal > operation.? However, consistency checks don't exclude blocks used for > swap, and they'll typically show mismatched blocks.If the swap is RAID1 on its own partitions (e.g., sda5/sdb5), then CHECK_DEVS in /etc/sysconfig/raid-check can be configured to check only specific devices. --keith -- kkeller at wombat.san-francisco.ca.us
> On 1/9/19 2:30 AM, Gary Stainburn wrote: >> 1) The big problem with this is that it is dependant on sda for booting. >> I >> did find an aritcle on how to set up boot loading on multiple HDD's, >> including cloning /boot/efi but I now can't find it. Does anyone know >> of a >> similar article? > > > Use RAID1 for /boot/efi as well.? The installer should get the details > right.Are you sure? How is the EFI firmware going to know about the RAID1? RAID1 is going to have type FD00 (Linux RAID) whereas EFI firmware expects type EF00 (EFI System Partition) to boot from. EFI then reads the GPT table directly and looks for a vfat filesystem to boot from. Default Linux software RAID1 on EL7 uses metadata 1.2 which is located at the beginning of the partition. EFI won't recognize the vfat filesystem behind the RAID metadata. Maybe certain EFI firmware is more tolerant but at least in my case I didn't get it to work on RAID1 at all. I'd really be interested if someone got it to work, how exactly it's configured. How exactly do the GPT tables look, how exactly is the RAID1 configured. Regards, Simon
I completed the install as described below which went well. I then copied the contents of /boot/efi to /boot/efi_copy using cd /boot rsync -acr efi/ efi_copy I then installed grub2 on sdb yum install yum install grub2-efi-modules grub2-install /dev/sdb everything ran as expected with no errors. The instructions then said to test the install by powering off and swapping sda and sdb. Instead, I just unplugged sda. When I booted the system loaded straight into a grub> prompt. I powered off and reconnected sda hoping to be able to continue experimenting. However, even with sda reconnected it just boots into a grub> prompt. Anyone got suggestions what to do next? (This server is not live, so can be wiped / experimented on) On Wednesday 09 January 2019 10:30:54 Gary Stainburn wrote:> I've just finished installing a new Bacula storeage server. Prior to doing > the install I did some research and ended up deciding to do the following > config. > > 6x4TB drives > /boot/efi efi_fs sda1 > /boot/efi_copy efi_fs sdb1 > /boot xfs RAID1 sda2 sdb2 > VG RAID6 all drives containing > SWAP > / > /home > /var/bacula > > Questions: > > 1) The big problem with this is that it is dependant on sda for booting. I > did find an aritcle on how to set up boot loading on multiple HDD's, > including cloning /boot/efi but I now can't find it. Does anyone know of a > similar article? > > 2) is putting SWAP in a RAID a good idea? Will it help, will it cause > problems? > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-- Gary Stainburn Group I.T. Manager Ringways Garages http://www.ringways.co.uk
> I completed the install as described below which went well. > > I then copied the contents of /boot/efi to /boot/efi_copy using > > cd /boot > rsync -acr efi/ efi_copy > > I then installed grub2 on sdb > > yum install yum install grub2-efi-modules > grub2-install /dev/sdb > > everything ran as expected with no errors. The instructions then said to > test > the install by powering off and swapping sda and sdb. Instead, I just > unplugged sda. > > When I booted the system loaded straight into a grub> prompt. > > I powered off and reconnected sda hoping to be able to continue > experimenting. > However, even with sda reconnected it just boots into a grub> prompt. > > Anyone got suggestions what to do next?If you boot, istn't there a way to get into an EFI boot menu where you can see the different EFI boot entries? Maybe you can see there what is wrong now. On the other hand, maybe you could even reinstall and try to configure /boot/efi on RAID1 as it was described by others in this thread? Regards, Simon
On 1/11/19 3:22 AM, Gary Stainburn wrote:> I then installed grub2 on sdb > > yum install yum install grub2-efi-modules > grub2-install /dev/sdbIf you want to manually provide redundant EFI system partitions (no RAID1), all you need to do is run efibootmgr to add a second entry.? You don't need to install grub2-efi-modules or run grub2-install.? Use: efibootmgr -c -w -L 'CentOS Linux' -d /dev/sdb -p 1 -l 'EFI\centos\shim.efi' ... where the option to -d is the name of the drive containing the EFI system partition and the option to -p is the EFI system partition number. For reference, that command will appear in /var/log/anaconda/program.log after any normal installation.