Hi, I have two servers and an iSCSI storage box I want to use to host Xen VMs on (making live migration possible). I have setup clustering LVM, made a logical volume, made a GFS filesystem on it and installed a stage3 Gentoo on it. I have made an initrd with the GFS kernel module on it and am trying to boot the VM. However I get this error upon booting: Mounting root filesystem ReiserFS: hda1: warning: sh-2021: reiserfs_fill_super: can not find reiserfs on hda1 mount: error 22 mounting reiserfs pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount /initrd/proc failed: 2 Freeing unused kernel memory: 144k freed Kernel panic - not syncing: No init found. Try passing init= option to kernel. Why is it trying to mount it as reiserfs when it is in fact gfs? Any way to make it mount it as gfs? And is my approach correct, or should I be doing this differently? -- Anders -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y? ------END GEEK CODE BLOCK------ PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>Why is it trying to mount it as reiserfs when it is in fact gfs? >Any way to make it mount it as gfs?Does it say it is GFS in fstab ? Anyway, how are you supposed to mount your GFS volume if your VM have not joined a GFS/CLUSTER network yet (or you disable the locking method?). Regards, Christophe PAINCHAUD _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Dec 09, 2005 at 09:40:34AM +0100, PAINCHAUD Christophe wrote:> >Why is it trying to mount it as reiserfs when it is in fact gfs? > >Any way to make it mount it as gfs? > Does it say it is GFS in fstab ? Anyway, how are you supposed to mount yourI does not reach the point where it even knows about fstab - it is when the kernel tries to mount the root partition.> GFS volume if your VM have not joined a GFS/CLUSTER network yet (or you > disable the locking method?).I thought about that too (and thinking about it again, I can see where I might have been too optimistic about this setup) - my two dom0 machines both run the gfs/clustering software, so I basically assumed that dom0 would handle all locking. That is of course not realistic, I can see that now. Since the root partitions for my virtual machines doesn''t need to be mounted in more than one place at a time, I guess I don''t really need a clustering filesystem on them. Am I correct? How about live migration, is it okay to just have the blockdevice available on both machines at the same time, or does that require a clustering filesystem? Again, my setup is with an iscsi storage box (an IBM DS300) which both machines has access to. If I don''t run a clustering filesystem and mount the same device on both machines, I run into trouble (obviously). But I am not sure if a partition containing a VM actually needs to be mounted two places at the same time, or if it just really quickly unmounts on the first dom0 and mounts it on the second? -- Anders -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y? ------END GEEK CODE BLOCK------ PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Dec 09, 2005 at 10:07:38AM +0100, Anders Bruun Olsen wrote:> Again, my setup is with an iscsi storage box (an IBM DS300) which both > machines has access to. If I don''t run a clustering filesystem and mount > the same device on both machines, I run into trouble (obviously). But I > am not sure if a partition containing a VM actually needs to be mounted > two places at the same time, or if it just really quickly unmounts on > the first dom0 and mounts it on the second?Forgot to mention that I am using Xen 3.0.0. -- Anders -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y? ------END GEEK CODE BLOCK------ PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Hi,>How about live migration, is it okay to just have the blockdevice >available on both machines at the same time, or does that require a >clustering filesystem?That''s what I am using : I dont have iSCSI devices, so I am using GNBD (part of redhat GFS/Cluster suite) to share block devices between all dom0. It works almost the same as your iSCSI configuration.>Again, my setup is with an iscsi storage box (an IBM DS300) which both >machines has access to. If I don''t run a clustering filesystem and mount >the same device on both machines, I run into trouble (obviously). But I >am not sure if a partition containing a VM actually needs to be mounted >two places at the same time, or if it just really quickly unmounts on >the first dom0 and mounts it on the second?You don''t need to mount any filesystem at all in dom0. dom0 needs to mount them when you are using loop devices (ie: a big file on dom0 mounted filesystem that you are using as a block for your domU) All what you need is to import your iSCSI block devices on all your real hosts and make sure they all have the same name on all of them (ie: /dev/myISCSI/vm001 ...). When you will start your VM on one of your dom0, it will share the block device with the domU. The domU itself will mount it. Then if you migrate your domU to another real host host, it will be paused, IO flushed and its memory and cpu registers transferred to the new host and unpaused, then it will resume its disks operations there. On the old real host, all resources have been freed so nothing should use the block device anymore. Also I have noticed when I ran the same VM twice on different real hosts sharing the same block device (formatted with ext3), that the second VM won''t boot until the first one has released the filesystem. It looks like EXT3 got a little mechanism that avoids you to mount it unless you use the force options (which a normal boot doesn''t do). So if you are using EXT3, and you make a mistake and run same VM on different xen nodes, there is a good probability that you won''t break anything. Regards, Christophe PAINCHAUD _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Friday 09 December 2005 10:09, Anders Bruun Olsen wrote:> On Fri, Dec 09, 2005 at 10:07:38AM +0100, Anders Bruun Olsen wrote: > > Again, my setup is with an iscsi storage box (an IBM DS300) which both > > machines has access to. If I don''t run a clustering filesystem and mount > > the same device on both machines, I run into trouble (obviously). But I > > am not sure if a partition containing a VM actually needs to be mounted > > two places at the same time, or if it just really quickly unmounts on > > the first dom0 and mounts it on the second? > > Forgot to mention that I am using Xen 3.0.0.Hi, theoretically, GFS would work in the case that you have a clustered filesystem ,accessible from your two dom0s, and you store the file-backed VBDs of your domUs on it, so you don''t need cluster-awareness within your domUs. Unfortunately, GFS doesn''t support, AFAIK, r/w mount of files on GFS over the loopback-device, which Xen uses to make the VBDs accessible for the domUs. If you tried to start a domU with that setup, you would get lots of i/o errors within your domUs. There''s a thread on this list about that and I made the same experience. Maybe OCFS2 would do a better job at this point. Using partitions of your iscsi box as VBDs for domUs should work fine considering live migrations. You don''t have to use a cluster filesystem, just use e.g. ext3 on a LV of your iscsi-box, put the domU root fs on it, make sure you have access to it from both your dom0s, be sure to not have mounted it from somewhere and start your domUs. I have a setup running with Xen 3.0 (testing, release of Monday which you can get from the website), the VBDs stored on LVs over fibre channel in my san, and migration works nice. For testing purposes I just run a test, domU is busy with i/o tests etc., and was migrated approx 400 times in the last 20 hours - still running :) Cheers, Michael -- ---------------------------------------------------------------------------------------- Michael Mey Thinking Objects Software GmbH | mailto: michael.mey@to.com Lilienthalstrasse 2/1 | phone: +49 711 88770-147 70825 Stuttgart-Korntal, Germany | fax: +49 711 88770-449 ---------------------------------------------------------------------------------------- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, 9 Dec 2005, Anders Bruun Olsen wrote:> Since the root partitions for my virtual machines doesn''t need to be > mounted in more than one place at a time, I guess I don''t really need a > clustering filesystem on them. Am I correct? How about live migration, > is it okay to just have the blockdevice available on both machines at > the same time, or does that require a clustering filesystem? > > Again, my setup is with an iscsi storage box (an IBM DS300) which both > machines has access to. If I don''t run a clustering filesystem and mount > the same device on both machines, I run into trouble (obviously). But I > am not sure if a partition containing a VM actually needs to be mounted > two places at the same time, or if it just really quickly unmounts on > the first dom0 and mounts it on the second?What I''m doing is using CLVM on the Xen dom0''s, and creating LV''s (formatted as XFS) for each of the domU''s. I then have a few GFS file systems (also LV''s) that get shared between some of the domU''s, but only ones that need access to them. Works quite well. ------------------------------------------------------------------------ | nate carlson | natecars@natecarlson.com | http://www.natecarlson.com | | depriving some poor village of its idiot since 1981 | ------------------------------------------------------------------------ _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Dec 09, 2005 at 10:29:07AM +0100, Michael Mey wrote:> > > Again, my setup is with an iscsi storage box (an IBM DS300) which both > > > machines has access to. If I don''t run a clustering filesystem and mount > > > the same device on both machines, I run into trouble (obviously). But I > > > am not sure if a partition containing a VM actually needs to be mounted > > > two places at the same time, or if it just really quickly unmounts on > > > the first dom0 and mounts it on the second? > > Forgot to mention that I am using Xen 3.0.0. > theoretically, GFS would work in the case that you have a clustered > filesystem ,accessible from your two dom0s, and you store the file-backed > VBDs of your domUs on it, so you don''t need cluster-awareness within your > domUs. > Unfortunately, GFS doesn''t support, AFAIK, r/w mount of files on GFS over the > loopback-device, which Xen uses to make the VBDs accessible for the domUs. > If you tried to start a domU with that setup, you would get lots of i/o errors > within your domUs. There''s a thread on this list about that and I made the > same experience. > Maybe OCFS2 would do a better job at this point.I tried OCFS2 earlier, but could not get it to work, the nodes in the cluster could not see each other. But that''s off-topic here :)> Using partitions of your iscsi box as VBDs for domUs should work fine > considering live migrations. You don''t have to use a cluster filesystem, just > use e.g. ext3 on a LV of your iscsi-box, put the domU root fs on it, make > sure you have access to it from both your dom0s, be sure to not have mounted > it from somewhere and start your domUs. > I have a setup running with Xen 3.0 (testing, release of Monday which you can > get from the website), the VBDs stored on LVs over fibre channel in my san, > and migration works nice. For testing purposes I just run a test, domU is > busy with i/o tests etc., and was migrated approx 400 times in the last 20 > hours - still running :)Thanks to all who answered, I now have it working (except I haven''t been able to test out live migration yet because of limitations at my end, but I am sure it will work nicely) with CLVMD and reiserfs. After getting a domU up and running with a Gentoo stage3 in it, I get the following error when it checks the root fs during boot: get_dma_info failed Bad file descriptor I have mapped the LV to hda1 because sda1 just gave an error. Anything that can be fixed or should I just ignore it? -- Anders -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCS/O d--@ s:+ a-- C++ UL+++$ P++ L+++ E- W+ N(+) o K? w O-- M- V PS+ PE@ Y+ PGP+ t 5 X R+ tv+ b++ DI+++ D+ G e- h !r y? ------END GEEK CODE BLOCK------ PGPKey: http://random.sks.keyserver.penguin.de:11371/pks/lookup?op=get&search=0xD4DEFED0 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users