Xen users - Mar 2008 - block-* scripts multiple issues

Greetings,

Let me preface this by saying I''m currently running Xen 3.2.0 under  
CentOS 5.1, although I believe these issues were present in 3.1.x also.


I''m experimenting with a setup that uses the block-drbd disk script  
provided by the drbd team. I''ve come across multiple problems, some of
which I believe would apply to any other block-* script...


1) The xm python libs (specifically the code path for
''create'') sanity
check the VBD to make sure it''s accessible. This is especially a  
problem with the drbd type since the ''device'' specification is
just a
label, so xm create has no idea how to find the real device. This I  
can work around with a quick patch that calls the drbdadm tool and  
gets the actual device name.

2) If you''re using pygrub, it is executed before any of the block  
scripts; so if there''s any setup that needs to occur before the VBD is
accessible, pygrub will fail. Again, this is the case with drbd  
because it won''t let you much around in the device if you
haven''t
become ''primary''. However, fixable with a workaround where you
specify
an alternate bootloader that''s just a wrapper around pygrub that does  
whatever magic you need.


And here''s the one I can''t work around...


3) If you use qemu-dm as the device model, it is started before the  
block scripts are run. So just like pygrub, it fails to access the VBD  
in some instances and then the VBD isn''t there in your domU. Which  
usually breaks your domU. This one I can''t just simply put a wrapper  
around since the VBD isn''t passed in on the command line.


It seems to me like it may make some sense to have an interface into  
the block scripts so xm create can say ''I need your real device name,  
and do any setup needed assuming you''re about to become
active''. This
would solve all three of the issues I listed, with a bit of added  
complication of how to pass the correct device name to qemu-dm.  
However, I don''t know the code well enough to know how the case of  
live migration/qemu-dm/block-* works and what (if anything) needs to  
be done there. I plan on testing that next.

Any ideas as to a reasonable solution to this stuff?

Thanks, and sorry about the length.

-=Zakk




_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I''m not sure why my mail server decided to suddenly push this through,
but whatever!

Since it''s out there, I''d like to follow up a bit and say I
DID manage
to work around the qemu-dm issue by wrapping it and doing some queries  
into the xenstore, but it''s quite ugly. I also patched the block-drbd  
scripts so that you specified the /dev/drbdX instead of the resource  
name, so any external utilities (like xm) that check the block device  
would work.

Migration works just fine; I think in that instance the block-*  
scripts run as part of the migration process and THEN qemu-dm starts?

-=Zakk

On Mar 5, 2008, at 11:39 AM, Zakk wrote:
> Greetings,
>
> Let me preface this by saying I''m currently running Xen 3.2.0
under
> CentOS 5.1, although I believe these issues were present in 3.1.x  
> also.
>
>
> I''m experimenting with a setup that uses the block-drbd disk
script
> provided by the drbd team. I''ve come across multiple problems,
some
> of which I believe would apply to any other block-* script...
[snip]

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Zakk <zakk-xenusers@rsdio.com> writes:
> I''m not sure why my mail server decided to suddenly push this
through,
> but whatever!
>
> Since it''s out there, I''d like to follow up a bit and say
I DID manage
> to work around the qemu-dm issue by wrapping it and doing some queries
> into the xenstore, but it''s quite ugly. I also patched the
block-drbd
> scripts so that you specified the /dev/drbdX instead of the resource
> name, so any external utilities (like xm) that check the block device
> would work.
>
> Migration works just fine; I think in that instance the block-*
> scripts run as part of the migration process and THEN qemu-dm
> starts?
Hmmm... qemu-dm is only used in fully virtualized domUs, right?

I''m having trouble with using drbd devices in xen 3.0.3, I''m
wondering
if it is the same issue.

Here are my symptoms:

if I define drbd device as 

disk = [ ''phy:/dev/drbd12,sda,w'' ] 

then it works, but then the device is not put into primary or
secondary mode automatically so the migration does not work

if I define it as 

disk = [ ''drbd:resourcename,sda,w'' ]

then I get

# xm create xxx -c
Using config file "./xxx".
Traceback (most recent call last):
  File "/usr/bin/pygrub", line 650, in ?
    fs = fsimage.open(file, get_fs_offset(file))
IOError: [Errno 95] Operation not supported
No handlers could be found for logger "xend"
Error: Boot loader didn''t return any data!

if I put a print file before line 650 in pygrub I see that the
resource name is being passed to pygrub instead of device name. If I
hardcode file in pygrub to device name then it works OK.

Is this a bug with xen 3.0.3 that is fixed later or will it work only
if I get the patches from you? ;)

-- 
To sto si frustriran, zavidan tko zna na cemu i sto ne vidis dalje od
svoje guzice je tuzno. Da onda barem imas toliko samokontrole da
sutis umjesto da pravis budalu od sebe... izgleda da si prestar da se
promjenis na bolje. - Davor Pasaric, hr.comp.mac


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Mar 12, 2008, at 12:11 PM, Aleksandar Ivanisevic wrote:
> Zakk <zakk-xenusers@rsdio.com> writes:
>
>> I''m not sure why my mail server decided to suddenly push this
>> through,
>> but whatever!
>>
>> Since it''s out there, I''d like to follow up a bit and
say I DID
>> manage
>> to work around the qemu-dm issue by wrapping it and doing some  
>> queries
>> into the xenstore, but it''s quite ugly. I also patched the
block-drbd
>> scripts so that you specified the /dev/drbdX instead of the resource
>> name, so any external utilities (like xm) that check the block device
>> would work.
>>
>> Migration works just fine; I think in that instance the block-*
>> scripts run as part of the migration process and THEN qemu-dm
>> starts?
>
> Hmmm... qemu-dm is only used in fully virtualized domUs, right?
>
> I''m having trouble with using drbd devices in xen 3.0.3,
I''m wondering
> if it is the same issue.
>
> Here are my symptoms:
*snip*

Your problem is related. The issue is that there''s a few things that  
try to access the block device before any of the block-* scripts are  
run. Pygrub is one of them. A quick fix is to specify a custom script  
as your bootloader and in that script determine which drbd resource is  
being used and explicitly set it primary. Then call the real pygrub.

That or don''t use pygrub at all and set kernel/initrd/etc in your domU
config file.

I did a bit of digging through all they xend/xm python code; but I''m  
not really sure how to solve this problem in the general case. I''m not
even 100% clear exactly how all those scripts get called; maybe when I  
have more time I''ll look at it again.



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Zakk <zakk-xenusers@rsdio.com> writes:

[...]
> I did a bit of digging through all they xend/xm python code; but
I''m
> not really sure how to solve this problem in the general case. I''m
not
> even 100% clear exactly how all those scripts get called; maybe when I
> have more time I''ll look at it again.
Same thing here. I''m actually wondering if there is anyone here who
has a working setup with drbd in primary/secondary mode that works
100% correctly, migration, pygrub and all...



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Dear Zakk,
could you share your solution? even if it''s ugly :)
Or maybe in the meantime it has become nicer!!

I need to use the DRBD wrapper for Xen on an HVM domain to better integrate
heartbeat.

Tnx in advance

Regards


Zakk-2 wrote:
> 
> I''m not sure why my mail server decided to suddenly push this
through,
> but whatever!
> 
> Since it''s out there, I''d like to follow up a bit and say
I DID manage
> to work around the qemu-dm issue by wrapping it and doing some queries  
> into the xenstore, but it''s quite ugly. I also patched the
block-drbd
> scripts so that you specified the /dev/drbdX instead of the resource  
> name, so any external utilities (like xm) that check the block device  
> would work.
> 
> Migration works just fine; I think in that instance the block-*  
> scripts run as part of the migration process and THEN qemu-dm starts?
> 
> -=Zakk
> 
> On Mar 5, 2008, at 11:39 AM, Zakk wrote:
> 
>> Greetings,
>>
>> Let me preface this by saying I''m currently running Xen 3.2.0
under
>> CentOS 5.1, although I believe these issues were present in 3.1.x  
>> also.
>>
>>
>> I''m experimenting with a setup that uses the block-drbd disk
script
>> provided by the drbd team. I''ve come across multiple problems,
some
>> of which I believe would apply to any other block-* script...
> 
> [snip]
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
-- 
View this message in context:
http://www.nabble.com/block-*-scripts-multiple-issues-tp15987535p24157167.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users