Re: [Xen-users] Xen & DRBD

Zakk
2008-03-18T09:06:36Z

block-drbd doesn''t really work. Well, it WORKS, but that''s
only half
the equation.

There are multiple things that try to access the DomU disk device/
files before the block/hotplug scripts are called. So far I know of:

"xm create": (if you specify a bootloader, it tries to verify the
block device is accessible, fails because there''s no valid block
device in drbd:resource0. It''ll try to open "resource0" and
fail)

pygrub: it has to read the root domU device; if this is a drbd
resource it gets passed the resource name and not the /dev/drbdX and
it fails. Even if it somehow got passed /dev/drbdX, it wouldn''t be
Primary and the open+read will fail.


qemu-dm: only relevant if you''re doing HVM. in that case your domU
will just not see the disk and fail in various ways.

In short, the only way I think it even has a hope of working without
lots of wrapping scripts and possibly patching some of the xen python
libraries is to do a para-virtualized domU, and specify the kernel/
initrd/etc in the domU config file. (In other words don''t use pygrub)

I''ve actually gotten it working by writing wrappers around qemu-dm,
pygrub and adding a bit of code to the xm command. It''s all very ugly
and not really the way to solve the problem. I think I''m just going to
have linux-ha change the disk state when needed, although I have to
verify migration will still work properly then.

-=Zak

On Mar 18, 2008, at 4:20 AM, Stuart Poulton wrote:

> Hi,
>
> I''ve just tried again, this time using one of the example xen
files.
> Details below.
>
>
> x-host-3:/etc/xen/vm # cat test
> # This is a bootloader used to boot paravirtualized domains.  You
> can optionally
> # plug in a different boot loader here, e.g., pygrub.  There is
> usually no
> # reason to change this.  Don''t explicitly specify kernel and
> ramdisk if you use
> # a bootloader.
> bootloader="/usr/lib/xen/boot/domUloader.py"
>
> # The domUloader will pull the specified kernel and initrd out of
> the domU''s
> # disk, and use that to boot.  This is easier to manage, compared to
> manually
> # keeping a copy of the kernel and initrd in sync in dom0''s
> filesystem and
> # pointing to them with the "kernel" and "ramdisk"
parameters.
> Syntax is
> # "VDEV:KERNEL,INITRD", where VDEV is the block device (from
domU''s
> point of
> # view) within which KERNEL and INITRD can be found.
> bootentry = "hda1:/boot/vmlinuz-xen,/boot/initrd-xen"
>
>
> # Remaining settings for the example VM:
> name="domUloader-example"
> memory=512
> vcpus=1
> on_crash="destroy"
> on_poweroff="destroy"
> on_reboot="restart"
> localtime=0
> builder="linux"
> bootargs="--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd-xen"
> disk=[ ''drbd:xendomains,xvda,w'' ]
> vif=[ ''mac=00:16:3e:00:01:02,bridge=xenbr0'' ]
> vfb=[''type=vnc,vncunused=1'']
>
>
> x-host-3:/etc/xen/vm # cat /proc/drbd
> version: 8.0.6 (api:86/proto:86)
> SVN Revision: 3048 build by phil@mescal, 2007-09-03 10:39:27
> 0: cs:Connected st:Primary/Primary ds:UpToDate/UpToDate C r---
>    ns:0 nr:1052672 dw:1052672 dr:0 al:0 bm:284 lo:0 pe:0 ua:0 ap:0
>        resync: used:0/31 hits:65650 misses:142 starving:0 dirty:0
> changed:142
>        act_log: used:0/257 hits:0 misses:0 starving:0 dirty:0
> changed:0
> x-host-3:/etc/xen/vm # drbdadm state xendomains
> Primary/Primary
>
>
> x-host-3:/etc/xen/vm # xm create -c test
> Using config file "./test".
> Error: Disk isn''t accessible
>
>
> x-host-3:/etc/xen/vm # xm create -n opensuse-1
> Using config file "./opensuse-1".
> (vm
>    (name opensuse-1)
>    (memory 384)
>    (on_poweroff destroy)
>    (on_reboot restart)
>    (on_crash destroy)
>    (vcpus 1)
>    (on_xend_start ignore)
>    (on_xend_stop ignore)
>    (uuid 7bf59311-0304-d34e-09a7-4d2f2fa5f5e9)
>    (bootloader /usr/lib/xen/boot/domUloader.py)
>    (bootloader_args ''--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd-
> xen'')
>    (image (linux (args '' '')))
>    (device (vbd (uname drbd:drdb0) (dev xvda) (mode w)))
>    (device (vbd (uname phy:/dev/cdrom) (dev xvdb:cdrom) (mode r)))
>    (device (vif (mac 00:16:3e:19:69:48)))
>    (device (vkbd))
>    (device
>        (vfb
>            (vncunused 1)
>            (type vnc)
>            (display localhost:10.0)
>            (xauthority /root/.Xauthority)
>        )
>    )
> )
>
>
>
> On Mon, 2008-03-17 at 19:28 +0000, Gareth Bult wrote:
>> Hi,
>>
>> When you switch to "drbd:" you need to change the path to the
drbd
>> resource name, rather than the device name. So if you have a
>> drbd.conf that says;
>>
>> resource fred {
>>   on nodea {
>>       device /dev/drbd0;
>>
>> You would use;
>> disk=[ ''drbd0:fred,xvda,w'',]
>>
>> hth
>> Gareth.
>>
>>
>> ----- Original Message -----
>> From: "Stuart Poulton" <swp@astro.le.ac.uk>
>> To: xen-users@lists.xensource.com
>> Sent: Monday, March 17, 2008 3:08:47 PM GMT +00:00 GMT Britain,
>> Ireland, Portugal
>> Subject: [Xen-users] Xen & DRBD
>>
>> Dear All,
>>
>> I''m using drbd to provide a HA pair of servers to run xen
domains on.
>>
>> I''ve created a test domain as follows
>>
>> x-host-2:~ # more /etc/xen/vm/opensuse
>> name="opensuse"
>> ostype="opensuse"
>> uuid="ee62226a-8487-cc02-d89f-dafafd1f35e6"
>> memory=1214
>> vcpus=2
>> on_crash="destroy"
>> on_poweroff="destroy"
>> on_reboot="restart"
>> localtime=0
>> builder="linux"
>> bootloader="/usr/lib/xen/boot/domUloader.py"
>> bootargs="--entry=xvda1:/boot/vmlinuz-xen,/boot/initrd-xen"
>> extra=" "
>> disk=[ ''phy:/dev/drbd0,xvda,w'',]
>> vif=[ ''mac=00:16:3e:29:fb:80'', ]
>> vfb=[''type=vnc,vncunused=1'']
>>
>> This works fine, and start and stop the domain on both of the nodes.
>> However if I try to use
>>
>> disk=[ ''drbd:xendomains,xvda,w'',]
>>
>> I get the following error....
>>
>> x-host-2:~ # xm create opensuse
>> Using config file "/etc/xen/vm/opensuse".
>> Error: Disk isn''t accessible
>>
>> I''m expecting the block-drbd script to do something here, but
from
>> what
>> I can tell it''s not.
>>
>> I suspect I''ve missed something fundamental. Any pointers
please ?
>>
>> Thanks
>>
>> Stuart
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users