tbrown@baremetal.com
2006-Jul-21 20:52 UTC
[Xen-users] HOWTO: AOE in domU and boot from it.
I know there was some discussion on this list a short while back where someone was advising putting the AOE "initiator" in dom0. I don''t like that solution for a number of reasons... two of them being that pushing the filesystem complexity into the domU makes the dom0 more stable _and_ makes domU migration trivial. I just finished spending at least 1/2 a day trying to get a simple initrd setup so I could boot a linux virtual machine via AOE. I had done it for nbd, and enbd, so I thought it would be fairly simple for AOE. Unfortunately, I could not figure out why the discovery process was not working. Turns out that I had forgotten that I needed to bring UP the ethernet interface first. it is covered on http://www.coraid.com/support/linux/contrib/vantuyl/aoeboot.html but that page is _way_ longer than it needs to be... at least for my taste. On a Centos (redhat enterprise linux clone) system, the steps to build an initrd to make it work are fairly simple. And are covered below: -------------------------------- We can leverage the system mkinitrd to great advantage. Basically, we run the system mkinitrd to generate a starting boot image, then we take it apart, make some small changes, and reassemble it. The changes we need are basically 3: 1) ifconfig eth0 up 2) create a /dev/etherd/discover device node 3) write to the discover node to trigger an immediate discover [root@xen4 ~/tmp]# uname -r 2.6.16-xen0 [root@xen4 ~/tmp]# mkinitrd --nocompress tmpx `uname -r` [root@xen4 ~/tmp]# mkdir initrd [root@xen4 ~/tmp]# cd initrd [root@xen4 ~/tmp/initrd]# cpio -i < ../tmpx 3144 blocks lets have a look! [root@xen4 ~/tmp/initrd]# ls -F bin/ dev/ etc/ init* lib/ loopfs/ proc/ sbin@ sys/ sysroot/ ok, we now have a directory tree with a good init file as a starting point and all the appropriate support utilities... way cool! We need to add busybox to our toolkit, because we need to run /bin/busybox ifconfig eth0 up to get the aoe discover to run. Use "yum install busybox" if the "which busybox" command below returns a "no busybox in ..." error [root@xen4 ~/tmp/initrd]# which busybox /sbin/busybox [root@xen4 ~/tmp/initrd]# ldd /sbin/busybox not a dynamic executable [root@xen4 ~/tmp/initrd]# cp /sbin/busybox bin/ That ldd check is just to prove it is a statically linked file, as dynamically linked files will not work in an initrd. We also need to modify the init script. OK. So my changes to the init file are to add the following commands ahead of the "Creating root device" line... since we need to have our root device available by the time we get to the mkrootdev line #TAB mods to get aoe up and running. # first, bring up the ethernet interface so aoe discover _can_ run echo /bin/busyybox ifconfig eth0 up /bin/busybox ifconfig eth0 up # give link a chance to autonegotiate etc... echo pausing to let ethernet settle sleep 2 #next, create device nodes to trigger discovery mkdir /dev/etherd /bin/busybox rm -f /dev/etherd/discover mknod /dev/etherd/discover c 152 3 echo touching /dev/etherd/discover echo hello world > /dev/etherd/discover echo pausing to aoe discovery run sleep 5 echo done aoe setup... cross your fingers echo #/TAB That fixes the init script. So we need to "burn" this into an image file we can use for an initrd... That''s easy, we''ll just copy the command used by the mkinitrd script find . | cpio -c -o | gzip -9 > /boot/img.aoe done. Boot it! My target environment is XEN, and the relevent parameters to be changed are: ramdisk = "/boot/img.aoe" root = "/dev/etherd/e9.0 ro" (or whatever your aoe target happens to be... I think that was the vblade default example). MODULES: I avoid using kernel modules to keep life simple, and using modules means you have to update the image file if you change kernels. If you want to use the kernel module, we can leverage mkinitrd for that too.. It just means you need to specify a "--preload aoe" option, and you have to be more carefull to get the kernel version parameter correct for your target kernel. Change the "mkinitrd --nocompress tmpx `uname -r`" command to be mkinitrd --nocompress --preload aoe tmpx `uname -r` or if you''re using a different target kernel, put that in where I have `uname -r` (but don''t use the back quotes... e.g. 2.6.16-xenU -Tom _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users