Dario Faggioli
2012-May-14 09:26 UTC
[PATCH v2] xl: introduce specific VCPU to PCPU mapping in config file
xm supports the following syntax (in the config file) for
specific VCPU to PCPU mapping:
cpus = ["2", "3"] # VCPU0 runs on CPU2, VCPU1 runs on CPU3
Allow for the same in xl.
Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com>
diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -108,9 +108,25 @@ created online and the remainder will be
=item B<cpus="CPU-LIST">
List of which cpus the guest is allowed to use. Default behavior is
-`all cpus`. A list of cpus may be specified as follows:
`cpus="0-3,5,^1"`
-(all vcpus will run on cpus 0,2,3,5), or `cpus=["2", "3"]`
(all vcpus
-will run on cpus 2 and 3).
+`all cpus`. A C<CPU-LIST> may be specified as follows:
+
+=over 4
+
+=item "all"
+
+To allow all the vcpus of the guest tov run on all the cpus on the host.
+
+=item "0-3,5,^1"
+
+To allow all the vcpus of the guest to run on cpus 0,2,3,5.
+
+=item [2, 3]
+
+To ask for specific vcpu mapping. That means (in this example), vcpu #0
+of the guest will run on cpu #2 of the host and vcpu #1 of the guest will
+run on cpu #3 of the host.
+
+=back
=item B<cpu_weight=WEIGHT>
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -71,6 +71,8 @@ static uint32_t domid;
static const char *common_domname;
static int fd_lock = -1;
+/* Stash for specific vcpu to pcpu mappping */
+static int *vcpu_to_pcpu;
static const char savefileheader_magic[32] "Xen saved domain, xl
format\n \0 \r";
@@ -630,6 +632,21 @@ static void parse_config_data(const char
exit(1);
}
+ /* Prepare the array for single vcpu to pcpu mappings */
+ vcpu_to_pcpu = xmalloc(sizeof(int) * b_info->max_vcpus);
+ memset(vcpu_to_pcpu, -1, sizeof(int) * b_info->max_vcpus);
+
+ /*
+ * Idea here is to let libxl think all the domain''s vcpus
+ * have cpu affinity with all the pcpus on the list.
+ * It is then us, here in xl, that matches each single vcpu
+ * to its pcpu (and that''s why we need to stash such info in
+ * the vcpu_to_pcpu array now) after the domain has been created.
+ * Doing it like this saves the burden of passing to libxl
+ * some big array hosting the single mappings. Also, using
+ * the cpumap derived from the list ensures memory is being
+ * allocated on the proper nodes anyway.
+ */
libxl_cpumap_set_none(&b_info->cpumap);
while ((buf = xlu_cfg_get_listitem(cpus, n_cpus)) != NULL) {
i = atoi(buf);
@@ -638,6 +655,8 @@ static void parse_config_data(const char
exit(1);
}
libxl_cpumap_set(&b_info->cpumap, i);
+ if (n_cpus < b_info->max_vcpus)
+ vcpu_to_pcpu[n_cpus] = i;
n_cpus++;
}
}
@@ -1714,6 +1733,31 @@ start:
if ( ret )
goto error_out;
+ /* If single vcpu to pcpu mapping was requested, honour it */
+ if (vcpu_to_pcpu) {
+ libxl_cpumap vcpu_cpumap;
+
+ libxl_cpumap_alloc(ctx, &vcpu_cpumap);
+ for (i = 0; i < d_config.b_info.max_vcpus; i++) {
+
+ if (vcpu_to_pcpu[i] != -1) {
+ libxl_cpumap_set_none(&vcpu_cpumap);
+ libxl_cpumap_set(&vcpu_cpumap, vcpu_to_pcpu[i]);
+ } else {
+ libxl_cpumap_set_any(&vcpu_cpumap);
+ }
+ if (libxl_set_vcpuaffinity(ctx, domid, i, &vcpu_cpumap)) {
+ fprintf(stderr, "setting affinity failed on vcpu
`%d''.\n", i);
+ libxl_cpumap_dispose(&vcpu_cpumap);
+ free(vcpu_to_pcpu);
+ ret = ERROR_FAIL;
+ goto error_out;
+ }
+ }
+ libxl_cpumap_dispose(&vcpu_cpumap);
+ free(vcpu_to_pcpu);
+ }
+
ret = libxl_userdata_store(ctx, domid, "xl",
config_data, config_len);
if (ret) {
Ian Campbell
2012-May-14 09:34 UTC
Re: [PATCH v2] xl: introduce specific VCPU to PCPU mapping in config file
On Mon, 2012-05-14 at 10:26 +0100, Dario Faggioli wrote:> xm supports the following syntax (in the config file) for > specific VCPU to PCPU mapping: > > cpus = ["2", "3"] # VCPU0 runs on CPU2, VCPU1 runs on CPU3 > > Allow for the same in xl. > > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> > > diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5 > --- a/docs/man/xl.cfg.pod.5 > +++ b/docs/man/xl.cfg.pod.5 > @@ -108,9 +108,25 @@ created online and the remainder will be > =item B<cpus="CPU-LIST"> > > List of which cpus the guest is allowed to use. Default behavior is > -`all cpus`. A list of cpus may be specified as follows: `cpus="0-3,5,^1"` > -(all vcpus will run on cpus 0,2,3,5), or `cpus=["2", "3"]` (all vcpus > -will run on cpus 2 and 3). > +`all cpus`. A C<CPU-LIST> may be specified as follows: > + > +=over 4 > + > +=item "all" > + > +To allow all the vcpus of the guest tov run on all the cpus on the host.typo: to> + > +=item "0-3,5,^1" > + > +To allow all the vcpus of the guest to run on cpus 0,2,3,5. > + > +=item [2, 3]Is this [2, 3] or ["2", "3"] as you used in the commit message? (would it be confusing the make those distinct, represent the current xl behaviour and xm behaviour respectively?) I think you missed my review on the v1 code when preparing this posting (we probably passed in mid-air)?> + > +To ask for specific vcpu mapping. That means (in this example), vcpu #0 > +of the guest will run on cpu #2 of the host and vcpu #1 of the guest will > +run on cpu #3 of the host. > + > +=back > > =item B<cpu_weight=WEIGHT> > > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c > --- a/tools/libxl/xl_cmdimpl.c > +++ b/tools/libxl/xl_cmdimpl.c > @@ -71,6 +71,8 @@ static uint32_t domid; > static const char *common_domname; > static int fd_lock = -1; > > +/* Stash for specific vcpu to pcpu mappping */ > +static int *vcpu_to_pcpu; > > static const char savefileheader_magic[32]> "Xen saved domain, xl format\n \0 \r"; > @@ -630,6 +632,21 @@ static void parse_config_data(const char > exit(1); > } > > + /* Prepare the array for single vcpu to pcpu mappings */ > + vcpu_to_pcpu = xmalloc(sizeof(int) * b_info->max_vcpus); > + memset(vcpu_to_pcpu, -1, sizeof(int) * b_info->max_vcpus); > + > + /* > + * Idea here is to let libxl think all the domain''s vcpus > + * have cpu affinity with all the pcpus on the list. > + * It is then us, here in xl, that matches each single vcpu > + * to its pcpu (and that''s why we need to stash such info in > + * the vcpu_to_pcpu array now) after the domain has been created. > + * Doing it like this saves the burden of passing to libxl > + * some big array hosting the single mappings. Also, using > + * the cpumap derived from the list ensures memory is being > + * allocated on the proper nodes anyway. > + */ > libxl_cpumap_set_none(&b_info->cpumap); > while ((buf = xlu_cfg_get_listitem(cpus, n_cpus)) != NULL) { > i = atoi(buf); > @@ -638,6 +655,8 @@ static void parse_config_data(const char > exit(1); > } > libxl_cpumap_set(&b_info->cpumap, i); > + if (n_cpus < b_info->max_vcpus) > + vcpu_to_pcpu[n_cpus] = i; > n_cpus++; > } > } > @@ -1714,6 +1733,31 @@ start: > if ( ret ) > goto error_out; > > + /* If single vcpu to pcpu mapping was requested, honour it */ > + if (vcpu_to_pcpu) { > + libxl_cpumap vcpu_cpumap; > + > + libxl_cpumap_alloc(ctx, &vcpu_cpumap); > + for (i = 0; i < d_config.b_info.max_vcpus; i++) { > + > + if (vcpu_to_pcpu[i] != -1) { > + libxl_cpumap_set_none(&vcpu_cpumap); > + libxl_cpumap_set(&vcpu_cpumap, vcpu_to_pcpu[i]); > + } else { > + libxl_cpumap_set_any(&vcpu_cpumap); > + } > + if (libxl_set_vcpuaffinity(ctx, domid, i, &vcpu_cpumap)) { > + fprintf(stderr, "setting affinity failed on vcpu `%d''.\n", i); > + libxl_cpumap_dispose(&vcpu_cpumap); > + free(vcpu_to_pcpu); > + ret = ERROR_FAIL; > + goto error_out; > + } > + } > + libxl_cpumap_dispose(&vcpu_cpumap); > + free(vcpu_to_pcpu); > + } > + > ret = libxl_userdata_store(ctx, domid, "xl", > config_data, config_len); > if (ret) {
Ian Jackson
2012-May-14 13:56 UTC
Re: [PATCH v2] xl: introduce specific VCPU to PCPU mapping in config file
Ian Campbell writes ("Re: [PATCH v2] xl: introduce specific VCPU to PCPU
mapping in config file"):> On Mon, 2012-05-14 at 10:26 +0100, Dario Faggioli wrote:
> > +=item "0-3,5,^1"
> > +
> > +To allow all the vcpus of the guest to run on cpus 0,2,3,5.
> > +
> > +=item [2, 3]
>
> Is this [2, 3] or ["2", "3"] as you used in the commit
message? (would
> it be confusing the make those distinct, represent the current xl
> behaviour and xm behaviour respectively?)
The current generic config parsing arrangements do not allow
config-item-specific code to distinguish between `"2"'' and
`2'' in the
config file. This would be possible in principle but let''s not do
this at this stage of the release.
I guess the docs should tell you to use numbers.
Ian.
Dario Faggioli
2012-May-14 17:03 UTC
Re: [PATCH v2] xl: introduce specific VCPU to PCPU mapping in config file
On Mon, 2012-05-14 at 10:34 +0100, Ian Campbell wrote:> On Mon, 2012-05-14 at 10:26 +0100, Dario Faggioli wrote: > > xm supports the following syntax (in the config file) for > > specific VCPU to PCPU mapping: > > > > cpus = ["2", "3"] # VCPU0 runs on CPU2, VCPU1 runs on CPU3 > > > > Allow for the same in xl. > > > > Signed-off-by: Dario Faggioli <dario.faggioli@citrix.com> > > > > ... > > + > > +=item "0-3,5,^1" > > + > > +To allow all the vcpus of the guest to run on cpus 0,2,3,5. > > + > > +=item [2, 3] > > Is this [2, 3] or ["2", "3"] as you used in the commit message? (would > it be confusing the make those distinct, represent the current xl > behaviour and xm behaviour respectively?) >Well, tools/examples/xmexample.hvm says xm supports `cpus=["2", "3"]`, while on xl both syntax (with or without the inner `"`) are recognized by xlu_get_list[item](). TBH, I don''t think it''s worth distinguishing between the two, especially considering we have the string syntax (i.e., `cpus="2,3") to achieve the current behaviour.> I think you missed my review on the v1 code when preparing this posting > (we probably passed in mid-air)? >Indeed! :-) Thanks and Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Dario Faggioli
2012-May-14 17:05 UTC
Re: [PATCH v2] xl: introduce specific VCPU to PCPU mapping in config file
On Mon, 2012-05-14 at 14:56 +0100, Ian Jackson wrote:> > Is this [2, 3] or ["2", "3"] as you used in the commit message? (would > > it be confusing the make those distinct, represent the current xl > > behaviour and xm behaviour respectively?) > > The current generic config parsing arrangements do not allow > config-item-specific code to distinguish between `"2"'' and `2'' in the > config file. This would be possible in principle but let''s not do > this at this stage of the release. >Agreed.> I guess the docs should tell you to use numbers. >I was thinking about mentioning both, as xm config examples uses the ["2", "3"] syntax... Is that reasonable? Thanks and Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel