This attached patch allows qemu to accept a drbd: prefix and interpret
it as a drbd resource. It then searches for it in /dev/drbd/by-res/
(which must be mapped via udev). It''s against 4.0-testing.
Also, the block scripts make the drbd resource primary but don''t want
for it to actually become available. A secondary drbd resource returns
EMEDIUMTYPE so if we get that result, wait half a second and retry, up
to a total of 5 retries. Fixing this in the block scripts might be
better but I can''t see how to do it cleanly.
James
diff --git a/xenstore.c b/xenstore.c
index 13b8d57..f6e2170 100644
--- a/xenstore.c
+++ b/xenstore.c
@@ -413,6 +413,9 @@ void xenstore_parse_domain_config(int hvm_domid)
char *danger_buf = NULL;
char *danger_type = NULL;
+ int retries = 1; /* number of attempts to open the block device -
drbd can take a second to become available */
+ int status;
+
for(i = 0; i < MAX_DRIVES + 1; i++)
media_filename[i] = NULL;
@@ -519,6 +522,14 @@ void xenstore_parse_domain_config(int hvm_domid)
format = &bdrv_raw;
}
+ if (!strcmp(drv, "drbd")) {
+ char *newparams = malloc(17 + strlen(params) + 1);
+ sprintf(newparams, "/dev/drbd/by-res/%s", params);
+ free(params);
+ params = newparams;
+ format = &bdrv_raw;
+ retries = 5;
+ }
#if 0
/* Phantom VBDs are disabled because the use of paths
* from guest-controlled areas in xenstore is unsafe.
@@ -617,7 +628,14 @@ void xenstore_parse_domain_config(int hvm_domid)
fprintf(stderr, "Using file %s in read-%s mode\n",
bs->filename, is_readonly ? "only" : "write");
- if (bdrv_open2(bs, params, flags, format) < 0)
+ do {
+ int status = bdrv_open2(bs, params, flags, format);
+ if (status != -EMEDIUMTYPE)
+ break;
+ usleep(500000); /* 500ms */
+ retries--;
+ } while (retries);
+ if (status < 0)
fprintf(stderr, "qemu: could not open vbd
''%s'' or hard
disk image ''%s'' (drv ''%s'' format
''%s'')\n", buf, params, drv ? drv : "?",
format ? format->format_name : "
}
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
James Harper writes ("[Xen-devel] [PATCH] make qemu handle drbd
properly"):> This attached patch allows qemu to accept a drbd: prefix and interpret
> it as a drbd resource. It then searches for it in /dev/drbd/by-res/
> (which must be mapped via udev). It''s against 4.0-testing.
Jim, did any of your drbd tests involve HVM domains ?
The code in qemu-xen in 4.0-testing is going to be very similar to
that in xen-unstable. I''m not sure whether we should be looking to
apply this to 4.1 ?
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Ian Jackson wrote:> James Harper writes ("[Xen-devel] [PATCH] make qemu handle drbd properly"): > >> This attached patch allows qemu to accept a drbd: prefix and interpret >> it as a drbd resource. It then searches for it in /dev/drbd/by-res/ >> (which must be mapped via udev). It''s against 4.0-testing. >> > > Jim, did any of your drbd tests involve HVM domains ? >No, they did not :(. But we have a similar hack (err, patch - attached for reference) for other external block scripts that we support. Interestingly, drbd is missing from the patch. I stumbled across this patch several months ago, but the original author is gone and I have not had time to consider a solution that will accommodate arbitrary external block scripts. Jim _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jim Fehlig writes ("Re: [Xen-devel] [PATCH] make qemu handle drbd
properly"):> Ian Jackson wrote:
> > Jim, did any of your drbd tests involve HVM domains ?
>
> No, they did not :(.
>
> But we have a similar hack (err, patch - attached for reference) for
> other external block scripts that we support. Interestingly, drbd is
> missing from the patch. I stumbled across this patch several months ago,
> but the original author is gone and I have not had time to consider a
> solution that will accommodate arbitrary external block scripts.
Hrm.
In 4.2, with the deprecation of xend, I think we are going to revisit
the way block devices are set up and things will become a lot simpler
and more tractable.
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-Jun-25 16:47 UTC
Re: [Xen-devel] [PATCH] make qemu handle drbd properly
On Wed, Jan 26, 2011 at 11:43 PM, James Harper < james.harper@bendigoit.com.au> wrote:> This attached patch allows qemu to accept a drbd: prefix and interpret > it as a drbd resource. It then searches for it in /dev/drbd/by-res/ > (which must be mapped via udev). It''s against 4.0-testing. > > Also, the block scripts make the drbd resource primary but don''t want > for it to actually become available. A secondary drbd resource returns > EMEDIUMTYPE so if we get that result, wait half a second and retry, up > to a total of 5 retries. Fixing this in the block scripts might be > better but I can''t see how to do it cleanly. > > James > > > diff --git a/xenstore.c b/xenstore.c > index 13b8d57..f6e2170 100644 > --- a/xenstore.c > +++ b/xenstore.c > @@ -413,6 +413,9 @@ void xenstore_parse_domain_config(int hvm_domid) > char *danger_buf = NULL; > char *danger_type = NULL; > > + int retries = 1; /* number of attempts to open the block device - > drbd can take a second to become available */ > + int status; > + > for(i = 0; i < MAX_DRIVES + 1; i++) > media_filename[i] = NULL; > > @@ -519,6 +522,14 @@ void xenstore_parse_domain_config(int hvm_domid) > format = &bdrv_raw; > } > > + if (!strcmp(drv, "drbd")) { > + char *newparams = malloc(17 + strlen(params) + 1); > + sprintf(newparams, "/dev/drbd/by-res/%s", params); > + free(params); > + params = newparams; > + format = &bdrv_raw; > + retries = 5; > + } > #if 0 > /* Phantom VBDs are disabled because the use of paths > * from guest-controlled areas in xenstore is unsafe. > @@ -617,7 +628,14 @@ void xenstore_parse_domain_config(int hvm_domid) > > fprintf(stderr, "Using file %s in read-%s mode\n", > bs->filename, is_readonly ? "only" : "write"); > > - if (bdrv_open2(bs, params, flags, format) < 0) > + do { > + int status = bdrv_open2(bs, params, flags, format); > + if (status != -EMEDIUMTYPE) > + break; > + usleep(500000); /* 500ms */ > + retries--; > + } while (retries); > + if (status < 0) > fprintf(stderr, "qemu: could not open vbd ''%s'' or hard > disk image ''%s'' (drv ''%s'' format ''%s'')\n", buf, params, drv ? drv : "?", > format ? format->format_name : " > } > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >Sorry to dig up an old thread. This patch doesnt seem to be in any of the qemu-xen-*, xen-*-testing or xen-unstable trees. Any reason why? shriram _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Shriram Rajagopalan writes ("Re: [Xen-devel] [PATCH] make qemu handle drbd
properly"):> On Wed, Jan 26, 2011 at 11:43 PM, James Harper <
> james.harper@bendigoit.com.au> wrote:
> > This attached patch allows qemu to accept a drbd: prefix and interpret
> > it as a drbd resource. It then searches for it in /dev/drbd/by-res/
> > (which must be mapped via udev). It''s against 4.0-testing.
...> Sorry to dig up an old thread. This patch doesnt seem to be in any of the
> qemu-xen-*, xen-*-testing or xen-unstable trees. Any reason why?
I don''t think we''re going to introduce a new feature of this
kind into
qemu-xen-unstable at this stage.
Sorry,
Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-Aug-30 04:11 UTC
Re: [Xen-devel] [PATCH] make qemu handle drbd properly
On Tue, Jun 28, 2011 at 8:54 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:> Shriram Rajagopalan writes ("Re: [Xen-devel] [PATCH] make qemu handle drbd properly"): >> On Wed, Jan 26, 2011 at 11:43 PM, James Harper < >> james.harper@bendigoit.com.au> wrote: >> > This attached patch allows qemu to accept a drbd: prefix and interpret >> > it as a drbd resource. It then searches for it in /dev/drbd/by-res/ >> > (which must be mapped via udev). It''s against 4.0-testing. > ... >> Sorry to dig up an old thread. This patch doesnt seem to be in any of the >> qemu-xen-*, xen-*-testing or xen-unstable trees. Any reason why? > > I don''t think we''re going to introduce a new feature of this kind into > qemu-xen-unstable at this stage. > > Sorry, > Ian. >Hi Ian, can you please include this patch into qemu-xen-unstable ? The last time I requested that this patch be included into qemu-xen-unstable, it was too late. I hope I am not late again this time. :) Please let me know if you need a more formal version of this patch. thanks shriram _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> -----Original Message----- > From: Shriram Rajagopalan [mailto:rshriram@cs.ubc.ca] > Sent: Tuesday, 30 August 2011 14:11 > To: Ian Jackson > Cc: James Harper; xen-devel@lists.xensource.com > Subject: Re: [Xen-devel] [PATCH] make qemu handle drbd properly > > On Tue, Jun 28, 2011 at 8:54 AM, Ian Jackson<Ian.Jackson@eu.citrix.com>> wrote: > > Shriram Rajagopalan writes ("Re: [Xen-devel] [PATCH] make qemuhandle drbd> properly"): > >> On Wed, Jan 26, 2011 at 11:43 PM, James Harper < > >> james.harper@bendigoit.com.au> wrote: > >> > This attached patch allows qemu to accept a drbd: prefix andinterpret> >> > it as a drbd resource. It then searches for it in/dev/drbd/by-res/> >> > (which must be mapped via udev). It''s against 4.0-testing. > > ... > >> Sorry to dig up an old thread. This patch doesnt seem to be in anyof the> >> qemu-xen-*, xen-*-testing or xen-unstable trees. Any reason why? > > > > I don''t think we''re going to introduce a new feature of this kindinto> > qemu-xen-unstable at this stage. > > > > Sorry, > > Ian. > > > > Hi Ian, > can you please include this patch into qemu-xen-unstable ? > The last time I requested that this patch be included intoqemu-xen-unstable,> it was too late. I hope I am not late again this time. :) > Please let me know if you need a more formal version of this patch. >While this patch works great, I''d like to see a more generic mechanism for passing different backends into qemu when they are fundamentally the same (just /dev entries) and the setup is handled by the block-* scripts. A block-iscsi would need the same futzing in qemu to make it happen properly. I think it''s wrong for qemu to use ''type'' - the access method should be stated explicitly. drbd/iscsi should be set up in block-drbd/iscsi and then qemu should handle them the same way it handles phy: (with the exception that drbd may need a delay while it comes online). So maybe another setting in the xenstore like qemu-type or access-method or something which, if present, overrides the type that qemu uses when deciding how to access the underlying block device. I don''t really like the name access-method, and qemu-type sounds wrong too, but can''t think of anything better. So the changes required would be: 1. The block-drbd script would need to write an entry access-method=phy into xenstore into the appropriate place. Maybe also write a flag in there to tell qemu that the device may take a few seconds to come online 2. qemu would need to first read access-method, and then type if access-type doesn''t exist. It would also retry the open of the device if the delay flag was specified. Or maybe it wouldn''t hurt to always retry. If the decision makers agree with that approach I think I could come up with a patch... James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Aug-30 10:36 UTC
RE: [Xen-devel] [PATCH] make qemu handle drbd properly
On Tue, 30 Aug 2011, James Harper wrote:> While this patch works great, I''d like to see a more generic mechanism > for passing different backends into qemu when they are fundamentally the > same (just /dev entries) and the setup is handled by the block-* > scripts. A block-iscsi would need the same futzing in qemu to make it > happen properly.I agree> I think it''s wrong for qemu to use ''type'' - the access method should be > stated explicitly. drbd/iscsi should be set up in block-drbd/iscsi and > then qemu should handle them the same way it handles phy: (with the > exception that drbd may need a delay while it comes online). So maybe > another setting in the xenstore like qemu-type or access-method or > something which, if present, overrides the type that qemu uses when > deciding how to access the underlying block device. I don''t really like > the name access-method, and qemu-type sounds wrong too, but can''t think > of anything better. > > So the changes required would be: > > 1. The block-drbd script would need to write an entry access-method=phy > into xenstore into the appropriate place. Maybe also write a flag in > there to tell qemu that the device may take a few seconds to come online > > 2. qemu would need to first read access-method, and then type if > access-type doesn''t exist. It would also retry the open of the device if > the delay flag was specified. Or maybe it wouldn''t hurt to always retry. > > If the decision makers agree with that approach I think I could come up > with a patch...Upstream qemu does not use xenstore to read the disk configuration and any mechanisms we introduce at this point should work on upstream qemu too. I don''t think qemu needs any knowledge of the backend type: the toolstack (block scripts and/or libxenlight) should just setup a phy backend and pass to qemu the right device name from the start. Regarding the retries, it is probably better to handle them in the block scripts, qemu doesn''t seem the right place to put them. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I don''t think qemu needs any knowledge of the backend type: the > toolstack (block scripts and/or libxenlight) should just setup a phy > backend and pass to qemu the right device name from the start. > Regarding the retries, it is probably better to handle them in theblock> scripts, qemu doesn''t seem the right place to put them.Does xen wait for the block-* scripts to complete before starting qemu? When I tested where to put the delay last time it didn''t seem to work without putting it in qemu-dm. James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Stefano Stabellini
2011-Aug-30 14:37 UTC
RE: [Xen-devel] [PATCH] make qemu handle drbd properly
On Tue, 30 Aug 2011, James Harper wrote:> > I don''t think qemu needs any knowledge of the backend type: the > > toolstack (block scripts and/or libxenlight) should just setup a phy > > backend and pass to qemu the right device name from the start. > > Regarding the retries, it is probably better to handle them in the > block > > scripts, qemu doesn''t seem the right place to put them. > > Does xen wait for the block-* scripts to complete before starting qemu? > When I tested where to put the delay last time it didn''t seem to work > without putting it in qemu-dm.Currently libxenlight does not support block-* script, but it should, it is one of the missing features. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel