Xen users - Jun 2006 - HA Xen on 2 servers!! No NFS, special hardware, DRBD or iSCSI...

I''ve been brainstorming...

I want to create a 2-node HA active/active cluster (In other words I want to run
a handful of
DomUs on one node and a handful on another).  In the event of a failure I want
all DomUs to fail
over to the other node and start working immediately.  I want absolutely no
single-points-of-failure.  I want to do it with free software and no special
hardware.  I want to
do it with just 2 servers.  I want it to be pretty simple to set up.  And I
would like to take
advantage Xen''s live migration feature for maintenance.

Not asking much, am I?  ;-)

This topic was discussed earlier this year (read the "Xen and iSCSI"
thread on the list archives)
and the best solution I saw was to create a poor-man''s SAN using DRBD,
Heartbeat, GFS and iSCSI on
two nodes and then run Xen on two MORE nodes.  Four nodes -- and quite a bit of
complication, and
I wonder what the performance was like?

I think I know how to do it with 2 nodes and (I think) it''ll be less
complicated and perform
better (I think).

I haven''t tried it, neither do I have any experience with AoE, LVM,
etc.  Only basic experience
with Xen.  But I think it should work.

Check it out:
* Get 2 computers with 2 NICs
* Install your favorite Linux distro on each
* Partition the drive into 4 partitions.  Two are Dom0 OS and swap, two
unformatted.
* Connect the intra-node NICs with a crossover/switch/hub (Didja know that
gigabit NICs are
auto-MDIX?  No crossover needed!)
* Configure the intra-node IPs to something like 10.0.0.1 and 10.0.0.2 or
192.168...
* Install Xen
* Install ATA over Ethernet and VBlade in Dom0
* Node1: vblade 0 1 /dev/hda3 eth1 # This is one of the unformatted partitions
* Node2: vblade 0 2 /dev/hda4 eth1 # The other
  # Or use vbladed but I don''t know how yet
* modprobe aoe on each node
* Install LVM on both nodes (in Dom0)
* Create two volume groups on each node:
  Node1: vgcreate DomU1hda /dev/hda3
  Node1: vgcreate DomU1hdb /dev/etherd/e0.2 # The AoE-exported device from the
other node
  Node2: vgcreate DomU2hda /dev/hda4
  Node2: vgcreate DomU1hdb /dev/etherd/e0.1
* Create logical volumes:
  Node1: lvcreate -n hda DomU1hda # Or do you create partitions here?  lvcreate
-n hda1?
  Node1: lvcreate -n hdb DomU1hdb
  Node2: lvcreate -n hda DomU1hda
  Node2: lvcreate -n hdb DomU1hdb
* fdisk /dev/DomU1hda/hda # Create two partitions for DomU1, OS and swap
* mkfs.ext3 /dev/DomU1hda/hda1 # Repeat for DomU1hdb and DomU2hdX
* mkswap /dev/DomU1hda/hda2    # Repeat for 
DomU1hdb and DomU2hdX
* Create a Xen DomU on each node with this configuration:
  Node1 DomU1:
     disk = [ ''phy:DomU1hda/hda1,hda1,w'' ]
     disk = [ ''phy:DomU1hdb/hdb1,hda1,w'' ]
     disk = [ ''phy:DomU1hda/hda2,hda2,w'' ]
     disk = [ ''phy:DomU1hdb/hdb2,hda2,w'' ]
  Node2 DomU2:
     disk = [ ''phy:DomU2hda/hda1,hda1,w'' ]
     disk = [ ''phy:DomU2hdb/hdb1,hda1,w'' ]
     disk = [ ''phy:DomU2hda/hda2,hda2,w'' ]
     disk = [ ''phy:DomU2hdb/hdb2,hda2,w'' ]
* Install the DomU OSes
* (Important part) Mirror the OSes using software RAID
* Install Heartbeat on both nodes in Dom0, ensure the Xen script uses live
migration when failing
over gracefully
* Run DomU1 on Node1, DomU2 on Node2

Result:
[       DomU1       ]   [       DomU2       ]
       /    \                  /    \
 [ hda ]     [ hdb ]     [ hda ]     [ hdb ]
       \    /                  \    /
 [       LVM       ]     [       LVM       ]
         /  \                         |    
[ Real HD ] [ AoE HD ]<--[ Real HD ]  |
                                   ___|
                                  /   |
           [ Real HD ]-->[ AoE HD ] [ Real HD ]
[       Node1        ]   [      Node2      ]



After a failure or during maintenance:
[       DomU1       ]   [       DomU2       ]
       /                       /
 [ hda ]                 [ hda ]
       \                        \
 [       LVM       ]     [       LVM       ]
         /               /
[ Real HD ]             /
                       /
                      /
           [ Real HD ]
[       Node1        ]



(ASCII art shore is purdy, Sam...)



LVM is not just a nice thing in this case, it is a necessity!  In addition to
being able to resize
the DomU''s partitions on the fly, it adds a layer of obfuscation so
that Xen is presented with the
same device name on both nodes.  I understand this is critical during a live
migration and appears
to be the reason for going with iSCSI or NFS.

The key is to use software mirroring within the DomU OSes.  I thought about
using DRBD alone but
that doesn''t allow live migration.  Works great if you do a regular
suspend-to-disk migration but
not live migration, because when you change a DRBD device from secondary to
primary you must
umount it (so you''d have to stop the DomU).

Mirroring also allows the DomU OS to restart if the host node it''s on
crashes because the data
should be consistent.  It also allows a DomU to keep operating -- no downtime --
if the other node
(the node it''s not running on) crashes.

Finally, AoE is painless to set up.  But I don''t see why AoE could not
be replaced with iSCSI if
it''s not working right.


I have three question marks in my head:
1.) How will it perform?
2.) Does it work?  Someone want to beat me to the punch and try it themselves? 
It''s likely to be
a little while before I can find the time to try it.
3.) Is it reliable?  Should be; AoE is relatively new but very simple.  LVM is
well-tested and
software mirroring is as old as the hills.


Thoughts?


CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> a little while before I can find the time to try it.
> 3.) Is it reliable?  Should be; AoE is relatively new but very simple.  LVM
is well-tested and
> software mirroring is as old as the hills.
>
>   
To confirm, you''re talking about doing software raid within the DomU?  
If one of the devices drops out, no big deal?  Interesting.  The biggest 
problem here when one of the devices becomes desyncronized, such as 
after a temporary network failure, you will rewrite the entire secondary 
device.

I have been thinking of testing NBD for a similar setup, instead of 
AoE.  My understanding is that NBD natively supports raid-1 mirroring 
with logic particularly useful for RAID-over-TCP.  That is, during a 
normal active-sync it should work just like regular software raid, but 
when the volumes become desynchronized (such as after a crash), it will 
not rewrite the whole disk from scratch (as is common in raid-1) but 
will only update inconsistent blocks (sort of like rsync).  NBD also 
supports Raid-5, btw.

Client-side RAID-1 of AoE devices will be problematic unless you use a 
driver modified in the above-described fashion.   I''m not (currently) 
aware of a driver that will solve that for you (but I also haven''t 
really looked).


--
Eric Windisch

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 07 June 2006 7:33 am, Eric Windisch wrote:
> I have been thinking of testing NBD for a similar setup, instead of
> AoE.  My understanding is that NBD natively supports raid-1 mirroring
> with logic particularly useful for RAID-over-TCP.  That is, during a
i think you''re mixing NBD and DRBD.

as i see it, DRBD lets you have two block devices, each one on it''s own
box,
and maintain them as mirrors.  any modification on one gets propagated to the 
other one.  now, if you want to make that "RAID1" available to a third
box
(and a fourth, fifth, etc) it''s usually shared with GNBD (but it could
be
AoE, iSCSI, ieee 1394, anything).

to Chris: yes, you could get what you want, just make a big data disk (or 
partition, or LVM PV) ''mirrored'' with DRBD, and use that for
the DomU''s
storage.  you''ll get live migration if the backing storage is
name-equivalent
on both boxes (easy to do with LVM).  if one hangs up, just reboot it on the 
other box.


-- 
Javier


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Eric Windisch <lists@bwbohh.net> wrote:
> > a little while before I can find the time to try it.
> > 3.) Is it reliable?  Should be; AoE is relatively new but very simple.
LVM is well-tested and
> > software mirroring is as old as the hills.
> >
> >   
> 
> To confirm, you''re talking about doing software raid within the
DomU?
Correct.

> If one of the devices drops out, no big deal?  Interesting.  The biggest 
> problem here when one of the devices becomes desyncronized, such as 
> after a temporary network failure, you will rewrite the entire secondary 
> device.
Bummer, you are correct.

> I have been thinking of testing NBD for a similar setup, instead of 
> AoE.  My understanding is that NBD natively supports raid-1 mirroring 
> with logic particularly useful for RAID-over-TCP.  That is, during a 
> normal active-sync it should work just like regular software raid, but 
> when the volumes become desynchronized (such as after a crash), it will 
> not rewrite the whole disk from scratch (as is common in raid-1) but 
> will only update inconsistent blocks (sort of like rsync).  NBD also 
> supports Raid-5, btw.
Yeah it uses the fast sync patch -- but that''s not specific to NBD, if
I recall; that is, one
could still use AoE and client-side software RAID with the fast sync patch and
have the same
benefit.  Downside is this is Linux-only.

If you don''t mind patching your DomU kernel (I prefer not to) with the
fast sync patch, this might
actually work.

> Client-side RAID-1 of AoE devices will be problematic unless you use a 
> driver modified in the above-described fashion.   I''m not
(currently)
> aware of a driver that will solve that for you (but I also haven''t
> really looked).
Yeah, looks like I can''t have my cake and eat it, too.  Either I use a
technology like DRBD on
Dom0 and do regular (non-live) migrations or use an external SAN or (perhaps)
patch my DomU kernel.

CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Javier Guerra <javier@guerrag.com> wrote:
> On Wednesday 07 June 2006 7:33 am, Eric Windisch wrote:
> > I have been thinking of testing NBD for a similar setup, instead of
> > AoE.  My understanding is that NBD natively supports raid-1 mirroring
> > with logic particularly useful for RAID-over-TCP.  That is, during a
> 
> i think you''re mixing NBD and DRBD.
No he got it right.  Similar technologies.  One advantage of NBD is it also
offers RAID 5.

> as i see it, DRBD lets you have two block devices, each one on
it''s own box,
> and maintain them as mirrors.  any modification on one gets propagated to
the
> other one.  now, if you want to make that "RAID1" available to a
third box
> (and a fourth, fifth, etc) it''s usually shared with GNBD (but it
could be
> AoE, iSCSI, ieee 1394, anything).
> 
> to Chris: yes, you could get what you want, just make a big data disk (or 
> partition, or LVM PV) ''mirrored'' with DRBD, and use that
for the DomU''s
> storage.  you''ll get live migration if the backing storage is
name-equivalent
> on both boxes (easy to do with LVM).  if one hangs up, just reboot it on
the
> other box.
Well that''s what I thought, but what happens if you want to do a live
migration and then take the
first node down for maintenance?

For example:
DRBD running as primary on Node1
DRBD partition exported to both boxes using GNBD (or iSCSI or AoE...)
Live migrate DomU1 from Node1 to Node2, which sets you up like this:
                [ DomU1 ]
                    |
  [ LVM  ]       [ LVM  ]
  [ GNBD ]------>[ GNBD ]
[ DRBD pri ]   [ DRBD sec ]
[  Node1   ]   [  Node2   ]

OK so far so good.

But now I want to take Node1 down for maintenance.  I cannot do that without
making the secondary
DRBD partition the new primary, which I cannot do without umounting the current
primary DRBD
partition, which I cannot do unless I suspend DomU1.

So as far as I can see, DRBD does not enable live migration + power-off first
node without some
sort of suspend/resume interruption.  My use of AoE would allow that but Eric is
right -- slight
network interruptions will cause a resync.

I wonder how often that would happen though, with a crossover cable?  Perhaps
it''s tolerable...
and if the guest OS is Linux, install the fast sync patch into the DomU...

I guess the regular migration suspend/resume interruption isn''t too
bad, I''m just reaching for the
holy grail ;-)

CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Javier Guerra wrote:
> On Wednesday 07 June 2006 7:33 am, Eric Windisch wrote:
>   
>> I have been thinking of testing NBD for a similar setup, instead of
>> AoE.  My understanding is that NBD natively supports raid-1 mirroring
>> with logic particularly useful for RAID-over-TCP.  That is, during a
>>     
>
> i think you''re mixing NBD and DRBD.
>
>   
I don''t think so.  I wasn''t previously aware of DRBD (but
I''ll certainly
look into it now).  I believe that NBD will do all that I described, at 
least according to: http://www.it.uc3m.es/~ptb/nbd/, but possibly not as 
well (but again, I''ll still need to look further into DRBD!)

--
Eric Windisch

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Chris de Vidal <chris@devidal.tv> wrote:
> If you don''t mind patching your DomU kernel (I prefer not to) with
the fast sync patch, this
> might actually work.
Excuse me, it''s called "Fast RAID 1" or fr1 and it''s
found here:
ftp://oboe.it.uc3m.es/pub/Programs/
It is "intelligent" in that it doesn''t blindly resynchronize
a whole mirror component when only a
few blocks need resyncing.  That can save hours of resync time on a large
device. In addition, it
always chooses the fastest device to read from on reads.


Exactly what we''re looking for!  Does not require NBD, from what I can
tell...


Reflecting upon this some more, I think perhaps it could be done in Dom0, making
setting up
software RAID inside the DomUs unnecessary.  But that requires a shared
hardware-like setup.  DRBD
0.7 doesn''t allow primary/primary (equivalent to shared hardware) but
things like AoE do (NBD
should, too).  When DRBD 0.8 becomes more stable and primary/primary is
possible, perhaps that
will be an option.  I like the DRBD project and would be eager to try it.

As Eric said, using AoE means any network interruption generates a resync.  But
then that''s a
concern with /ANY/ AoE or iSCSI + software RAID setup.  So methinks AoE
isn''t too bad.  Add in the
fr1 patch and it might be usable.

If one were to patch the kernel with the "Fast RAID" patch and use AoE
to create /dev/md0 on Dom0
using both a local disk and a remote, this might work!  In this case, LVM would
no longer be a
necessity.


Does anyone know if Xen''s live migration requires something like GFS on
shared hardware?  Or does
it take care of the problem as long as it''s the only thing accessing
the drive?

CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > Does anyone know if Xen''s live migration requires something
like GFS on
> > shared hardware?  Or does it take care of the problem as long as
it''s the
> > only thing accessing the drive?
> >
> I''ve had live migration working using iet/open-iscsi supplied vbds
formatted
> with reiserfs.
Cool, thanks for answering that.

I have a question about your setup: After you live-migrate can you then power
off the first node?

CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 07 June 2006 15:24, Chris de Vidal wrote:
> --- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > > Does anyone know if Xen''s live migration requires
something like GFS on
> > > shared hardware?  Or does it take care of the problem as long as
it''s
> > > the only thing accessing the drive?
> >
> > I''ve had live migration working using iet/open-iscsi supplied
vbds
> > formatted with reiserfs.
>
> Cool, thanks for answering that.
>
> I have a question about your setup: After you live-migrate can you then
> power off the first node?
>
I must admit it''s not something we explicitly tried, but I
don''t see why not.
Apart from the configuration file and the contents of the log files,
there''s
nothing on the node you''ve migrated from to suggest the domU ever
existed
there.

At the moment I''m just talking about a single iSCSI target machine.
What we
hope to get going soon is a mirrored storage system using drbd providing 
iscsi through ha-linux or possibly multipath-tools. Then I can have multiple 
xen dom0''s accessing the "disks" they need for the
domU''s. The main thing is
to have the vbd''s accessed through a persistent name and not just
/dev/sda
etc.

We still have to look at managing the xen vm configuration files, though they 
could be supplied through nfs from the NAS boxes.

Matthew
-- 
Matthew Wild                       Tel.: +44 (0)1235 445173
M.Wild@rl.ac.uk                    URL http://www.ukssdc.ac.uk/
UK Solar System Data Centre and
World Data Centre - Solar-Terrestrial Physics, Chilton
Rutherford Appleton Laboratory, Chilton, Didcot, Oxon, OX11 0QX

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > > I''ve had live migration working using iet/open-iscsi
supplied vbds
> > > formatted with reiserfs.
> >
> > Cool, thanks for answering that.
> >
> > I have a question about your setup: After you live-migrate can you
then
> > power off the first node?
> >
> I must admit it''s not something we explicitly tried, but I
don''t see why not.
> Apart from the configuration file and the contents of the log files,
there''s
> nothing on the node you''ve migrated from to suggest the domU ever
existed
> there.
Now I''m scratching my brain to figure out just how that''s
possible... does it copy the entire DomU
over, including the hard drive?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 07 June 2006 16:19, Chris de Vidal wrote:
> --- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > > > I''ve had live migration working using
iet/open-iscsi supplied vbds
> > > > formatted with reiserfs.
> > >
> > > Cool, thanks for answering that.
> > >
> > > I have a question about your setup: After you live-migrate can
you then
> > > power off the first node?
> >
> > I must admit it''s not something we explicitly tried, but I
don''t see why
> > not. Apart from the configuration file and the contents of the log
files,
> > there''s nothing on the node you''ve migrated from to
suggest the domU ever
> > existed there.
>
> Now I''m scratching my brain to figure out just how that''s
possible... does
> it copy the entire DomU over, including the hard drive?
No, the "hard drive" for the domU is provided via iSCSI from a storage
server
box. As long as the domU configuration points to a vbd with the same scsi 
label on each dom0 box, and they''re online, xen can handle the
migration.

Matthew
-- 
Matthew Wild                       Tel.: +44 (0)1235 445173
M.Wild@rl.ac.uk                    URL http://www.ukssdc.ac.uk/
UK Solar System Data Centre and
World Data Centre - Solar-Terrestrial Physics, Chilton
Rutherford Appleton Laboratory, Chilton, Didcot, Oxon, OX11 0QX

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

--- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > Now I''m scratching my brain to figure out just how
that''s possible... does
> > it copy the entire DomU over, including the hard drive?
> 
> No, the "hard drive" for the domU is provided via iSCSI from a
storage server
> box. As long as the domU configuration points to a vbd with the same scsi 
> label on each dom0 box, and they''re online, xen can handle the
migration.
Ohhhhh, you use three-four boxes.  I''m talking about doing this with
just two, and I think it''s doable.

CD

TenThousandDollarOffer.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,
> Ohhhhh, you use three-four boxes.  I''m talking about doing this
with just two, and I think it''s doable.
How did that go? Did you get it all going on just two boxes?

Cheers.
Chris.

On 6/7/06, Chris de Vidal <chris@devidal.tv>
wrote:
> --- Matthew Wild <M.Wild@rl.ac.uk> wrote:
> > > Now I''m scratching my brain to figure out just how
that''s possible... does
> > > it copy the entire DomU over, including the hard drive?
> >
> > No, the "hard drive" for the domU is provided via iSCSI from
a storage server
> > box. As long as the domU configuration points to a vbd with the same
scsi
> > label on each dom0 box, and they''re online, xen can handle
the migration.
>
> Ohhhhh, you use three-four boxes.  I''m talking about doing this
with just two, and I think it''s doable.
>
> CD
>
> TenThousandDollarOffer.com
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Chris de Vidal wrote:
> Reflecting upon this some more, I think perhaps it could be done in Dom0,
making setting up
> software RAID inside the DomUs unnecessary.  But that requires a shared
hardware-like setup.  DRBD
> 0.7 doesn''t allow primary/primary (equivalent to shared hardware)
but things like AoE do (NBD
> should, too).  When DRBD 0.8 becomes more stable and primary/primary is
possible, perhaps that
> will be an option.  I like the DRBD project and would be eager to try it.
> 
> As Eric said, using AoE means any network interruption generates a resync. 
But then that''s a
> concern with /ANY/ AoE or iSCSI + software RAID setup.  So methinks AoE
isn''t too bad.  Add in the
> fr1 patch and it might be usable.
> 
> If one were to patch the kernel with the "Fast RAID" patch and
use AoE to create /dev/md0 on Dom0
> using both a local disk and a remote, this might work!  In this case, LVM
would no longer be a
> necessity.
It can''t see how this should work with RAID in Dom0. At that point you 
are just at the same problem as with DRBD. During live migration, you 
would first have to deactivate the RAID on the first node and only then 
activate it on the second node.

But for live migration to work, both need to be active for a short 
period of time.

I wonder if anyone will ever step up and create a patch to xend that 
makes it possible to do live migration without having a storage setup 
that can be active at two nodes, i.e. suspend -> disconnect storage -> 
connect storage on second node -> resume on second node. I am really 
sorry I have no time to do it myself.

 > Does anyone know if Xen''s live migration requires something like
GFS
 > on shared hardware?  Or does
 > it take care of the problem as long as it''s the only thing
accessing
 > the drive?

Only one node must access the device at a time. But for your 
RAID-in-dom0 idea, that is not the case.

Best Regards,
Michael Paesold






_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I''ve tried this a few ways, and found there is really no
"good" way that
fits every need that I found. This is, for lack of a better phrase , a
pain in the ass. 

You can do it similar to how it was described if using a san, you form
and break the md raid on the san triggered via [ insert favorite
interconnect method here ]. That''s tricky, and if not timed right
creates issues. I''ve gotten this working with iscsi in the past, but
I''ve also run into major meltdowns.

You can also use something like gfs, but if you have high performance
I/O needs, well.. it can hinder things a bit.

You also need to take into consideration the power going out at the
worst possible time, and what would happen if it did [meltdown,
usually].

Your best bet (for now) is to use something like Open SSI with 2.0.7
(paravirtualized) , or 3.0.2 with open SSI as a hvm if you want a true
HA setup to get it rolled out quick and avoid (many) 3 AM phone calls,
unless this is also a learning and tinkering experience for you.

Open SSI is very easy to get going, and does a very good job. Xen lends
even more management and much faster boot / recovery to the setup. You
can then just migrate the entire virtualized cluster, or a cluster node
setup using etherboot over to another machine and avoid the hassles of
disk sync all together (for all but the director nodes). If done as HVM
with 3.0.2, you can do some really neat dynamic scaling as well.
Problems with this - bridging gets to be a nightmare, and dom-0 needs to
be a little ''fatter'' than usual. 

To my knowledge, nobody has ported the Xen/SSI paravirtualized kernel
over to 3.0.2, however I could be wrong. I''m also pretty sure the
whitebox flavor of SSI was never ported to Xen, but again, I could be
wrong. 

I know it sounds a little convoluted and complex, but it does solve a
bunch of problems regarding this.

HTH
-Tim

On Wed, 2006-08-30 at 14:53 +0200, Michael Paesold
wrote:
> Chris de Vidal wrote:
> > Reflecting upon this some more, I think perhaps it could be done in
Dom0, making setting up
> > software RAID inside the DomUs unnecessary.  But that requires a
shared hardware-like setup.  DRBD
> > 0.7 doesn''t allow primary/primary (equivalent to shared
hardware) but things like AoE do (NBD
> > should, too).  When DRBD 0.8 becomes more stable and primary/primary
is possible, perhaps that
> > will be an option.  I like the DRBD project and would be eager to try
it.
> > 
> > As Eric said, using AoE means any network interruption generates a
resync.  But then that''s a
> > concern with /ANY/ AoE or iSCSI + software RAID setup.  So methinks
AoE isn''t too bad.  Add in the
> > fr1 patch and it might be usable.
> > 
> > If one were to patch the kernel with the "Fast RAID" patch
and use AoE to create /dev/md0 on Dom0
> > using both a local disk and a remote, this might work!  In this case,
LVM would no longer be a
> > necessity.
> 
> It can''t see how this should work with RAID in Dom0. At that point
you
> are just at the same problem as with DRBD. During live migration, you 
> would first have to deactivate the RAID on the first node and only then 
> activate it on the second node.
> 
> But for live migration to work, both need to be active for a short 
> period of time.
> 
> I wonder if anyone will ever step up and create a patch to xend that 
> makes it possible to do live migration without having a storage setup 
> that can be active at two nodes, i.e. suspend -> disconnect storage
->
> connect storage on second node -> resume on second node. I am really 
> sorry I have no time to do it myself.
> 
>  > Does anyone know if Xen''s live migration requires something
like GFS
>  > on shared hardware?  Or does
>  > it take care of the problem as long as it''s the only thing
accessing
>  > the drive?
> 
> Only one node must access the device at a time. But for your 
> RAID-in-dom0 idea, that is not the case.
> 
> Best Regards,
> Michael Paesold
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users