Hello, I spoke briefly to Ians C and J about this at FOSDEM and they suggested I send an email about it. Behaviour through at least Xen 3.3 to 4.0 on domU restart (i.e., when root inside domU does "restart" or "shutdown -r") is to restart the domU without re-reading the domU config file. As a result, domU will start up again with only the RAM, block devices, network routes, etc. that it had when it was last started. A common sequence of events here is: - User submits support ticket asking for an extra block device, a block of IP addresses routed to them, etc. - We give them that without shutting their domU down, because there is no need to shut it down. We tell them, "next time you need to reboot, please take care to actually shut it down and boot it again instead, otherwise you''ll lose what we just gave you." - Later -- often much later -- when they need to reboot for some reason, the above comment has gone completely out of their heads and they reboot. - domU comes back without the disk, route, whatever they got used to having, and completely breaks things for them. - More support tickets and disruption because of a reboot that didn''t go as they expected. No amount of documentation or teaching seems to help as this is unintuitive behaviour compared to what normally happens when an admin on bare metal does a reboot. We''re actually on the verge of forcing a shut down just to avoid this misunderstanding, without any technical requirement for one. Either that or some horrible script that watches logs for domain startup and then checks they have all the things they are meant to have. As far as I am aware I can set on_restart to something other than "restart" but there is currently no option that makes it re-read the config file and restart. I understand this will never be fixed in xend/xm, but would it be possible to have xl re-read the domU''s config when the domU restarts? Thanks, Andy
Andy Smith writes ("[Xen-devel] Re-reading domain configs on domain
restart"):> I understand this will never be fixed in xend/xm, but would it be
> possible to have xl re-read the domU''s config when the domU
> restarts?
Yes, this would certainly be possible. Straightforward, even.
There are two questions we need to answer first:
Firstly, how should this be requested ? I think a command-line option
to xl would do the trick.
Secondly, should we change the default behaviour ? I''d be inclined
towards "yes" as the current arrangement is really rather perverse,
but I''m open to opinions.
In the longer term, in the spirit of xl, it might be a good idea to
give the user the option of running a hook script when the domain
dies/reboots/etc. This would simplify a number of things.
Ian.
Hi Andy, Good to meet you over the w.e. On Tue, 2012-02-07 at 17:30 +0000, Ian Jackson wrote:> Andy Smith writes ("[Xen-devel] Re-reading domain configs on domain restart"): > > I understand this will never be fixed in xend/xm, but would it be > > possible to have xl re-read the domU''s config when the domU > > restarts? > > Yes, this would certainly be possible. Straightforward, even. > There are two questions we need to answer first: > > Firstly, how should this be requested ? I think a command-line option > to xl would do the trick.Where would you give it? Giving it to create might be unhelpful if you only decide you need it when you come to reboot. Giving to reboot is tricky since then you need that xl to communicate with the one actually managing the reboots etc (we could deal with that though). Another option would be to allow it as an option to the on_FOO=bar in the configuration file. There''s a bit of a combinatorial explosion going on there though (coredump & X, rename & X etc) and it has the same problem with being too late at reboot time to change your mind.> Secondly, should we change the default behaviour ? I''d be inclined > towards "yes" as the current arrangement is really rather perverse, > but I''m open to opinions.I lean that way too, although if higher level tools using xl are e.g. writing to a /tmp file which they delete after start up then might that be an issue? Eventually I''d like to see a libxl function which can reconstitute a libxl_domain_config from a running domains, such that it is sufficient to pass to libxl_domain_create to recreate that domain. That needn''t block this functionality though.> In the longer term, in the spirit of xl, it might be a good idea to > give the user the option of running a hook script when the domain > dies/reboots/etc. This would simplify a number of things.Yes, hooks are a good idea.
On Wed, 2012-02-08 at 16:32 +0000, Ian Campbell wrote:> Hi Andy, > > Good to meet you over the w.e. > > On Tue, 2012-02-07 at 17:30 +0000, Ian Jackson wrote: > > Andy Smith writes ("[Xen-devel] Re-reading domain configs on domain restart"): > > > I understand this will never be fixed in xend/xm, but would it be > > > possible to have xl re-read the domU''s config when the domU > > > restarts? > > > > Yes, this would certainly be possible. Straightforward, even. > > There are two questions we need to answer first: > > > > Firstly, how should this be requested ? I think a command-line option > > to xl would do the trick.> > Secondly, should we change the default behaviour ? I''d be inclined > > towards "yes" as the current arrangement is really rather perverse, > > but I''m open to opinions.Did we decide what to do here? It occurred to me that rereading by default requires that the config file still exist when the reboot happens, which may not be the case for certain usecase which dump the config in a $TMPFILE or use the command line syntax (someone said they did this). I suppose we could fallback to the original config if the file no longer exists, but that would be potentially confusing. Also I wonder if people do xl create template.cfg name="VM1" vi template.cfg hack it around xl create template.cfg name="VM2" It would be odd for VM1 to turn into VM2 on reboot. Hrm, rereading also takes no account of options passed on the command line on reboot. As another (hopefully simple) idea how about a "xl dom-set-config" or similar which (only) updates the config stashed in the userinfo to be used on reboot? I would specifically exclude the possibility of this reconfiguring the running domain for simplicity. Ian
Hi, On Thu, Feb 23, 2012 at 11:17:42AM +0000, Ian Campbell wrote:> As another (hopefully simple) idea how about a "xl dom-set-config" or > similar which (only) updates the config stashed in the userinfo to be > used on reboot? I would specifically exclude the possibility of this > reconfiguring the running domain for simplicity.As a user I would be happy with this solution as it still lets me do what I want to do. Cheers, Andy
On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote:> As another (hopefully simple) idea how about a "xl dom-set-config" or > similar which (only) updates the config stashed in the userinfo to be > used on reboot? I would specifically exclude the possibility of this > reconfiguring the running domain for simplicity.I''ve only lightly tested the following, but it seemed to do what I expected (I used it to change memory from 512 to 1024 for a Windows VM on reboot). I think there might be a better name, that better reflects the fact that it doesn''t actually change the config right now, any ideas? "domain-config-override"? Thoughts on the general concept? Ian. 8<------------------------------------------------------- xl: provide a command to set the saved configuration for a running domain Pickup this new configuration on reboot. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 8b82b7435b7d -r 063d98ba8309 docs/man/xl.pod.1 --- a/docs/man/xl.pod.1 Thu Feb 23 12:31:18 2012 +0000 +++ b/docs/man/xl.pod.1 Thu Feb 23 15:14:29 2012 +0000 @@ -148,6 +148,31 @@ soon as it is run. =back +=item B<config-update> B<domid> [I<configfile>] [I<OPTIONS>] + +Update the saved configuration for a running domain. This has no +immediate effect but will be applied when the guest is next +restarted. This command is useful to ensure that runtime modifications +made to the guest will be preserved when the guest is restarted. + +I<configfile> has to be an absolute path to a file. + +B<OPTIONS> + +=over 4 + +=item B<-f=FILE>, B<--defconfig=FILE> + +Use the given configuration file. + +=item B<key=value> + +It is possible to pass I<key=value> pairs on the command line to provide +options as if they were written in the configuration file; these override +whatever is in the I<configfile>. + +=back + =item B<console> [I<OPTIONS>] I<domain-id> Attach to domain I<domain-id>''s console. If you''ve set up your domains to diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl.h --- a/tools/libxl/xl.h Thu Feb 23 12:31:18 2012 +0000 +++ b/tools/libxl/xl.h Thu Feb 23 15:14:29 2012 +0000 @@ -50,6 +50,7 @@ int main_reboot(int argc, char **argv); int main_list(int argc, char **argv); int main_list_vm(int argc, char **argv); int main_create(int argc, char **argv); +int main_config_update(int argc, char **argv); int main_button_press(int argc, char **argv); int main_vcpupin(int argc, char **argv); int main_vcpuset(int argc, char **argv); diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl_cmdimpl.c --- a/tools/libxl/xl_cmdimpl.c Thu Feb 23 12:31:18 2012 +0000 +++ b/tools/libxl/xl_cmdimpl.c Thu Feb 23 15:14:29 2012 +0000 @@ -1201,11 +1201,31 @@ skip_vfb: xlu_cfg_destroy(config); } +static void reload_domain_config(libxl_ctx *ctx, uint32_t domid, + uint8_t **config_data, int *config_len) +{ + uint8_t *t_data; + int ret, t_len; + + ret = libxl_userdata_retrieve(ctx, domid, "xl", &t_data, &t_len); + if (ret) { + LOG("failed to retrieve guest configuration (rc=%d). " + "reusing old configuration", ret); + return; + } + + free(*config_data); + *config_data = t_data; + *config_len = t_len; +} + /* Returns 1 if domain should be restarted, * 2 if domain should be renamed then restarted, or 0 */ static int handle_domain_death(libxl_ctx *ctx, uint32_t domid, libxl_event *event, + uint8_t **config_data, int *config_len, libxl_domain_config *d_config) + { int restart = 0; libxl_action_on_shutdown action; @@ -1260,10 +1280,13 @@ static int handle_domain_death(libxl_ctx break; case LIBXL_ACTION_ON_SHUTDOWN_RESTART_RENAME: + reload_domain_config(ctx, domid, config_data, config_len); restart = 2; break; case LIBXL_ACTION_ON_SHUTDOWN_RESTART: + reload_domain_config(ctx, domid, config_data, config_len); + restart = 1; /* fall-through */ case LIBXL_ACTION_ON_SHUTDOWN_DESTROY: @@ -1748,7 +1771,9 @@ start: LOG("Domain %d has shut down, reason code %d 0x%x", domid, event->u.domain_shutdown.shutdown_reason, event->u.domain_shutdown.shutdown_reason); - switch (handle_domain_death(ctx, domid, event, &d_config)) { + switch (handle_domain_death(ctx, domid, event, + (uint8_t **)&config_data, &config_len, + &d_config)) { case 2: if (!preserve_domain(ctx, domid, event, &d_config)) { /* If we fail then exit leaving the old domain in place. */ @@ -1785,6 +1810,12 @@ start: d_config.c_info.name = strdup(common_domname); } + /* Reparse the configuration in case it has changed */ + libxl_domain_config_dispose(&d_config); + memset(&d_config, 0, sizeof(d_config)); + parse_config_data(config_file, config_data, config_len, + &d_config); + /* * XXX FIXME: If this sleep is not there then domain * re-creation fails sometimes. @@ -3394,6 +3425,120 @@ int main_create(int argc, char **argv) return 0; } +int main_config_update(int argc, char **argv) +{ + const char *filename = NULL; + char *p; + char extra_config[1024]; + void *config_data = 0; + int config_len = 0; + libxl_domain_config d_config; + int opt, rc; + int option_index = 0; + int debug = 0; + static struct option long_options[] = { + {"help", 0, 0, ''h''}, + {"defconfig", 1, 0, ''f''}, + {0, 0, 0, 0} + }; + + if (argc < 2) { + fprintf(stderr, "xl config-update requires a domain argument\n"); + help("config-update"); + exit(1); + } + + find_domain(argv[1]); + argc--; argv++; + + if (argv[1] && argv[1][0] != ''-'' && !strchr(argv[1], ''='')) { + filename = argv[1]; + argc--; argv++; + } + + while (1) { + opt = getopt_long(argc, argv, "dhqf:", long_options, &option_index); + if (opt == -1) + break; + + switch (opt) { + case ''d'': + debug = 1; + break; + case ''f'': + filename = optarg; + break; + case ''h'': + help("create"); + return 0; + default: + fprintf(stderr, "option `%c'' not supported.\n", optopt); + break; + } + } + + extra_config[0] = ''\0''; + for (p = extra_config; optind < argc; optind++) { + if (strchr(argv[optind], ''='') != NULL) { + p += snprintf(p, sizeof(extra_config) - (p - extra_config), + "%s\n", argv[optind]); + } else if (!filename) { + filename = argv[optind]; + } else { + help("create"); + return 2; + } + } + if (filename) { + free(config_data); config_data = 0; + rc = libxl_read_file_contents(ctx, filename, + &config_data, &config_len); + if (rc) { fprintf(stderr, "Failed to read config file: %s: %s\n", + filename, strerror(errno)); return ERROR_FAIL; } + if (strlen(extra_config)) { + if (config_len > INT_MAX - (strlen(extra_config) + 2 + 1)) { + fprintf(stderr, "Failed to attach extra configration\n"); + exit(1); + } + /* allocate space for the extra config plus two EOLs plus \0 */ + config_data = realloc(config_data, config_len + + strlen(extra_config) + 2 + 1); + if (!config_data) { + fprintf(stderr, "Failed to realloc config_data\n"); + exit(1); + } + config_len += sprintf(config_data + config_len, "\n%s\n", + extra_config); + } + } else { + fprintf(stderr, "Config file not specified\n"); + exit(1); + } + + memset(&d_config, 0x00, sizeof(d_config)); + + parse_config_data(filename, config_data, config_len, &d_config); + + if (debug || dryrun_only) + printf_info(default_output_format, -1, &d_config); + + if (!dryrun_only) { + fprintf(stderr, "setting dom%d configuration\n", domid); + rc = libxl_userdata_store(ctx, domid, "xl", + config_data, config_len); + if (rc) { + fprintf(stderr, "failed to update configuration\n"); + exit(1); + } + } + + libxl_domain_config_dispose(&d_config); + + free(config_data); + + return 0; +} + static void button_press(const char *p, const char *b) { libxl_trigger trigger; @@ -3917,7 +4062,6 @@ static int sched_credit_domain_set( { int rc; - rc = libxl_sched_credit_domain_set(ctx, domid, scinfo); if (rc) fprintf(stderr, "libxl_sched_credit_domain_set failed.\n"); diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl_cmdtable.c --- a/tools/libxl/xl_cmdtable.c Thu Feb 23 12:31:18 2012 +0000 +++ b/tools/libxl/xl_cmdtable.c Thu Feb 23 15:14:29 2012 +0000 @@ -32,6 +32,15 @@ struct cmd_spec cmd_table[] = { "-d Enable debug messages.\n" "-e Do not wait in the background for the death of the domain." }, + { "config-update", + &main_config_update, 1, + "Update a running domain''s saved configuration, used when rebuilding " + "the domain after reboot", + "<Domain> <ConfigFile> [options] [vars]", + "-h Print this help.\n" + "-f FILE, --defconfig=FILE\n Use the given configuration file.\n" + "-d Enable debug messages.\n" + }, { "list", &main_list, 0, "List information about all/some domains",
Ian Campbell writes ("Re: [Xen-devel] Re-reading domain configs on domain
restart"):> On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote:
> > As another (hopefully simple) idea how about a "xl
dom-set-config" or
> > similar which (only) updates the config stashed in the userinfo to be
> > used on reboot? I would specifically exclude the possibility of this
> > reconfiguring the running domain for simplicity.
>
> I''ve only lightly tested the following, but it seemed to do what I
> expected (I used it to change memory from 512 to 1024 for a Windows VM
> on reboot).
>
> I think there might be a better name, that better reflects the fact that
> it doesn''t actually change the config right now, any ideas?
> "domain-config-override"?
>
> Thoughts on the general concept?
This looks good to me. I''m tempted to just apply it but we can wait a
bit more for comments.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Ian.
On Thu, Feb 23, 2012 at 3:16 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote: >> As another (hopefully simple) idea how about a "xl dom-set-config" or >> similar which (only) updates the config stashed in the userinfo to be >> used on reboot? I would specifically exclude the possibility of this >> reconfiguring the running domain for simplicity. > > I''ve only lightly tested the following, but it seemed to do what I > expected (I used it to change memory from 512 to 1024 for a Windows VM > on reboot). > > I think there might be a better name, that better reflects the fact that > it doesn''t actually change the config right now, any ideas? > "domain-config-override"? > > Thoughts on the general concept?Is the config used for anything other than a reboot? If so, it might lead to unexpected behavior, if for example, I want to change config option X on reboot, but it''s read (and acted on) before I get done with my various changes. It would be good if the user could be sure that none of the changes would have any effect until the next reboot. I think the concept of having a "config-on-reboot" is a good one. For convenience, would it make sense to add a parameter to "xl reboot" to take an argument (-c for example) that will do the config override before issuing the reboot command? -George> > Ian. > 8<------------------------------------------------------- > > xl: provide a command to set the saved configuration for a running domain > > Pickup this new configuration on reboot. > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > > diff -r 8b82b7435b7d -r 063d98ba8309 docs/man/xl.pod.1 > --- a/docs/man/xl.pod.1 Thu Feb 23 12:31:18 2012 +0000 > +++ b/docs/man/xl.pod.1 Thu Feb 23 15:14:29 2012 +0000 > @@ -148,6 +148,31 @@ soon as it is run. > > =back > > +=item B<config-update> B<domid> [I<configfile>] [I<OPTIONS>] > + > +Update the saved configuration for a running domain. This has no > +immediate effect but will be applied when the guest is next > +restarted. This command is useful to ensure that runtime modifications > +made to the guest will be preserved when the guest is restarted. > + > +I<configfile> has to be an absolute path to a file. > + > +B<OPTIONS> > + > +=over 4 > + > +=item B<-f=FILE>, B<--defconfig=FILE> > + > +Use the given configuration file. > + > +=item B<key=value> > + > +It is possible to pass I<key=value> pairs on the command line to provide > +options as if they were written in the configuration file; these override > +whatever is in the I<configfile>. > + > +=back > + > =item B<console> [I<OPTIONS>] I<domain-id> > > Attach to domain I<domain-id>''s console. If you''ve set up your domains to > diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl.h > --- a/tools/libxl/xl.h Thu Feb 23 12:31:18 2012 +0000 > +++ b/tools/libxl/xl.h Thu Feb 23 15:14:29 2012 +0000 > @@ -50,6 +50,7 @@ int main_reboot(int argc, char **argv); > int main_list(int argc, char **argv); > int main_list_vm(int argc, char **argv); > int main_create(int argc, char **argv); > +int main_config_update(int argc, char **argv); > int main_button_press(int argc, char **argv); > int main_vcpupin(int argc, char **argv); > int main_vcpuset(int argc, char **argv); > diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl_cmdimpl.c > --- a/tools/libxl/xl_cmdimpl.c Thu Feb 23 12:31:18 2012 +0000 > +++ b/tools/libxl/xl_cmdimpl.c Thu Feb 23 15:14:29 2012 +0000 > @@ -1201,11 +1201,31 @@ skip_vfb: > xlu_cfg_destroy(config); > } > > +static void reload_domain_config(libxl_ctx *ctx, uint32_t domid, > + uint8_t **config_data, int *config_len) > +{ > + uint8_t *t_data; > + int ret, t_len; > + > + ret = libxl_userdata_retrieve(ctx, domid, "xl", &t_data, &t_len); > + if (ret) { > + LOG("failed to retrieve guest configuration (rc=%d). " > + "reusing old configuration", ret); > + return; > + } > + > + free(*config_data); > + *config_data = t_data; > + *config_len = t_len; > +} > + > /* Returns 1 if domain should be restarted, > * 2 if domain should be renamed then restarted, or 0 */ > static int handle_domain_death(libxl_ctx *ctx, uint32_t domid, > libxl_event *event, > + uint8_t **config_data, int *config_len, > libxl_domain_config *d_config) > + > { > int restart = 0; > libxl_action_on_shutdown action; > @@ -1260,10 +1280,13 @@ static int handle_domain_death(libxl_ctx > break; > > case LIBXL_ACTION_ON_SHUTDOWN_RESTART_RENAME: > + reload_domain_config(ctx, domid, config_data, config_len); > restart = 2; > break; > > case LIBXL_ACTION_ON_SHUTDOWN_RESTART: > + reload_domain_config(ctx, domid, config_data, config_len); > + > restart = 1; > /* fall-through */ > case LIBXL_ACTION_ON_SHUTDOWN_DESTROY: > @@ -1748,7 +1771,9 @@ start: > LOG("Domain %d has shut down, reason code %d 0x%x", domid, > event->u.domain_shutdown.shutdown_reason, > event->u.domain_shutdown.shutdown_reason); > - switch (handle_domain_death(ctx, domid, event, &d_config)) { > + switch (handle_domain_death(ctx, domid, event, > + (uint8_t **)&config_data, &config_len, > + &d_config)) { > case 2: > if (!preserve_domain(ctx, domid, event, &d_config)) { > /* If we fail then exit leaving the old domain in place. */ > @@ -1785,6 +1810,12 @@ start: > d_config.c_info.name = strdup(common_domname); > } > > + /* Reparse the configuration in case it has changed */ > + libxl_domain_config_dispose(&d_config); > + memset(&d_config, 0, sizeof(d_config)); > + parse_config_data(config_file, config_data, config_len, > + &d_config); > + > /* > * XXX FIXME: If this sleep is not there then domain > * re-creation fails sometimes. > @@ -3394,6 +3425,120 @@ int main_create(int argc, char **argv) > return 0; > } > > +int main_config_update(int argc, char **argv) > +{ > + const char *filename = NULL; > + char *p; > + char extra_config[1024]; > + void *config_data = 0; > + int config_len = 0; > + libxl_domain_config d_config; > + int opt, rc; > + int option_index = 0; > + int debug = 0; > + static struct option long_options[] = { > + {"help", 0, 0, ''h''}, > + {"defconfig", 1, 0, ''f''}, > + {0, 0, 0, 0} > + }; > + > + if (argc < 2) { > + fprintf(stderr, "xl config-update requires a domain argument\n"); > + help("config-update"); > + exit(1); > + } > + > + find_domain(argv[1]); > + argc--; argv++; > + > + if (argv[1] && argv[1][0] != ''-'' && !strchr(argv[1], ''='')) { > + filename = argv[1]; > + argc--; argv++; > + } > + > + while (1) { > + opt = getopt_long(argc, argv, "dhqf:", long_options, &option_index); > + if (opt == -1) > + break; > + > + switch (opt) { > + case ''d'': > + debug = 1; > + break; > + case ''f'': > + filename = optarg; > + break; > + case ''h'': > + help("create"); > + return 0; > + default: > + fprintf(stderr, "option `%c'' not supported.\n", optopt); > + break; > + } > + } > + > + extra_config[0] = ''\0''; > + for (p = extra_config; optind < argc; optind++) { > + if (strchr(argv[optind], ''='') != NULL) { > + p += snprintf(p, sizeof(extra_config) - (p - extra_config), > + "%s\n", argv[optind]); > + } else if (!filename) { > + filename = argv[optind]; > + } else { > + help("create"); > + return 2; > + } > + } > + if (filename) { > + free(config_data); config_data = 0; > + rc = libxl_read_file_contents(ctx, filename, > + &config_data, &config_len); > + if (rc) { fprintf(stderr, "Failed to read config file: %s: %s\n", > + filename, strerror(errno)); return ERROR_FAIL; } > + if (strlen(extra_config)) { > + if (config_len > INT_MAX - (strlen(extra_config) + 2 + 1)) { > + fprintf(stderr, "Failed to attach extra configration\n"); > + exit(1); > + } > + /* allocate space for the extra config plus two EOLs plus \0 */ > + config_data = realloc(config_data, config_len > + + strlen(extra_config) + 2 + 1); > + if (!config_data) { > + fprintf(stderr, "Failed to realloc config_data\n"); > + exit(1); > + } > + config_len += sprintf(config_data + config_len, "\n%s\n", > + extra_config); > + } > + } else { > + fprintf(stderr, "Config file not specified\n"); > + exit(1); > + } > + > + memset(&d_config, 0x00, sizeof(d_config)); > + > + parse_config_data(filename, config_data, config_len, &d_config); > + > + if (debug || dryrun_only) > + printf_info(default_output_format, -1, &d_config); > + > + if (!dryrun_only) { > + fprintf(stderr, "setting dom%d configuration\n", domid); > + rc = libxl_userdata_store(ctx, domid, "xl", > + config_data, config_len); > + if (rc) { > + fprintf(stderr, "failed to update configuration\n"); > + exit(1); > + } > + } > + > + libxl_domain_config_dispose(&d_config); > + > + free(config_data); > + > + return 0; > +} > + > static void button_press(const char *p, const char *b) > { > libxl_trigger trigger; > @@ -3917,7 +4062,6 @@ static int sched_credit_domain_set( > { > int rc; > > - > rc = libxl_sched_credit_domain_set(ctx, domid, scinfo); > if (rc) > fprintf(stderr, "libxl_sched_credit_domain_set failed.\n"); > diff -r 8b82b7435b7d -r 063d98ba8309 tools/libxl/xl_cmdtable.c > --- a/tools/libxl/xl_cmdtable.c Thu Feb 23 12:31:18 2012 +0000 > +++ b/tools/libxl/xl_cmdtable.c Thu Feb 23 15:14:29 2012 +0000 > @@ -32,6 +32,15 @@ struct cmd_spec cmd_table[] = { > "-d Enable debug messages.\n" > "-e Do not wait in the background for the death of the domain." > }, > + { "config-update", > + &main_config_update, 1, > + "Update a running domain''s saved configuration, used when rebuilding " > + "the domain after reboot", > + "<Domain> <ConfigFile> [options] [vars]", > + "-h Print this help.\n" > + "-f FILE, --defconfig=FILE\n Use the given configuration file.\n" > + "-d Enable debug messages.\n" > + }, > { "list", > &main_list, 0, > "List information about all/some domains", > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Fri, 2012-03-02 at 10:04 +0000, George Dunlap wrote:> On Thu, Feb 23, 2012 at 3:16 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote: > >> As another (hopefully simple) idea how about a "xl dom-set-config" or > >> similar which (only) updates the config stashed in the userinfo to be > >> used on reboot? I would specifically exclude the possibility of this > >> reconfiguring the running domain for simplicity. > > > > I''ve only lightly tested the following, but it seemed to do what I > > expected (I used it to change memory from 512 to 1024 for a Windows VM > > on reboot). > > > > I think there might be a better name, that better reflects the fact that > > it doesn''t actually change the config right now, any ideas? > > "domain-config-override"? > > > > Thoughts on the general concept? > > Is the config used for anything other than a reboot? If so, it might > lead to unexpected behavior, if for example, I want to change config > option X on reboot, but it''s read (and acted on) before I get done > with my various changes. It would be good if the user could be sure > that none of the changes would have any effect until the next reboot.The option takes a whole new configuration file so you are either using the old or new configuration. There should be no intermediate state. AFAIK this file is used for any domain "restart", which includes reboot, save+restore and migration. I think that any such changes which you might want to apply on reboot you would also want to apply post migration or restore too. I think the right long term solution to this is libxl_running_domain_to_config(ctx, domid, &libxl_domain_config) but that isn''t going to happen for 4.2 at least and this is a good bandaid plus a useful generic facility.> I think the concept of having a "config-on-reboot" is a good one. For > convenience, would it make sense to add a parameter to "xl reboot" to > take an argument (-c for example) that will do the config override > before issuing the reboot command?We have this for migrate and restore so I think extending that to reboot is a sensible idea. Ian.
On Fri, Mar 2, 2012 at 10:15 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2012-03-02 at 10:04 +0000, George Dunlap wrote: >> On Thu, Feb 23, 2012 at 3:16 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote: >> >> As another (hopefully simple) idea how about a "xl dom-set-config" or >> >> similar which (only) updates the config stashed in the userinfo to be >> >> used on reboot? I would specifically exclude the possibility of this >> >> reconfiguring the running domain for simplicity. >> > >> > I''ve only lightly tested the following, but it seemed to do what I >> > expected (I used it to change memory from 512 to 1024 for a Windows VM >> > on reboot). >> > >> > I think there might be a better name, that better reflects the fact that >> > it doesn''t actually change the config right now, any ideas? >> > "domain-config-override"? >> > >> > Thoughts on the general concept? >> >> Is the config used for anything other than a reboot? If so, it might >> lead to unexpected behavior, if for example, I want to change config >> option X on reboot, but it''s read (and acted on) before I get done >> with my various changes. It would be good if the user could be sure >> that none of the changes would have any effect until the next reboot. > > The option takes a whole new configuration file so you are either using > the old or new configuration. There should be no intermediate state. > > AFAIK this file is used for any domain "restart", which includes reboot, > save+restore and migration. > > I think that any such changes which you might want to apply on reboot > you would also want to apply post migration or restore too.Wait, what kind of con fig changes would you want to apply across save-restore? Memory is something that can''t be changed across save-restore, at least for an HVM domain. Changes to CPU feature availability or hardware availability (e.g., NX bit, acpi, &c) will almost certainly cause the guest to crash; I would think removing or changing devices (e.g., removing a NIC, or changing from one type to another) would have the same effect. The only config option that comes to mind one might wish to change is something like moving from a dom0 qemu to a stubdom; but that seems like a very peculiar use-case.> I think the right long term solution to this is > libxl_running_domain_to_config(ctx, domid, &libxl_domain_config) > but that isn''t going to happen for 4.2 at least and this is a good > bandaid plus a useful generic facility. > >> I think the concept of having a "config-on-reboot" is a good one. For >> convenience, would it make sense to add a parameter to "xl reboot" to >> take an argument (-c for example) that will do the config override >> before issuing the reboot command? > > We have this for migrate and restore so I think extending that to reboot > is a sensible idea.Right -- what kinds of things were envisioned to change on a migrate? -George> > Ian. > >
On Mon, 2012-03-05 at 06:06 -0500, George Dunlap wrote:> On Fri, Mar 2, 2012 at 10:15 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Fri, 2012-03-02 at 10:04 +0000, George Dunlap wrote: > >> On Thu, Feb 23, 2012 at 3:16 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote: > >> >> As another (hopefully simple) idea how about a "xl dom-set-config" or > >> >> similar which (only) updates the config stashed in the userinfo to be > >> >> used on reboot? I would specifically exclude the possibility of this > >> >> reconfiguring the running domain for simplicity. > >> > > >> > I''ve only lightly tested the following, but it seemed to do what I > >> > expected (I used it to change memory from 512 to 1024 for a Windows VM > >> > on reboot). > >> > > >> > I think there might be a better name, that better reflects the fact that > >> > it doesn''t actually change the config right now, any ideas? > >> > "domain-config-override"? > >> > > >> > Thoughts on the general concept? > >> > >> Is the config used for anything other than a reboot? If so, it might > >> lead to unexpected behavior, if for example, I want to change config > >> option X on reboot, but it''s read (and acted on) before I get done > >> with my various changes. It would be good if the user could be sure > >> that none of the changes would have any effect until the next reboot. > > > > The option takes a whole new configuration file so you are either using > > the old or new configuration. There should be no intermediate state. > > > > AFAIK this file is used for any domain "restart", which includes reboot, > > save+restore and migration. > > > > I think that any such changes which you might want to apply on reboot > > you would also want to apply post migration or restore too. > > Wait, what kind of con fig changes would you want to apply across save-restore?The original request IIRC was to preserve changes made to a running domain, such as adding a new VIF (using network-add) when the guest was eventually restarted by updating the configuration used on restart. This is useful if the restarts are out of your control (e.g. because the customer does them himself). Today the VM will return to its original state not the state right before the reboot. Especially annoying if this results in loss of N/W connectivity for the user etc.> Memory is something that can''t be changed across save-restore, at > least for an HVM domain. Changes to CPU feature availability or > hardware availability (e.g., NX bit, acpi, &c) will almost certainly > cause the guest to crash;Yes, you probably wouldn''t want to be changing these using this mechanism apart from on reboot.> I would think removing or changing devices > (e.g., removing a NIC, or changing from one type to another) would > have the same effect.At least for PV devices this will be fine (modulo kernel bugs), a migration looks pretty much like a complete teardown and rebuild of the PV devices. It''s also not a problem in the case of wanting to preserve manually made changes on reboot which is the main motivation here. Ian.
On Thu, 2012-03-01 at 18:40 +0000, Ian Jackson wrote:> Ian Campbell writes ("Re: [Xen-devel] Re-reading domain configs on domain restart"): > > On Thu, 2012-02-23 at 11:17 +0000, Ian Campbell wrote: > > > As another (hopefully simple) idea how about a "xl dom-set-config" or > > > similar which (only) updates the config stashed in the userinfo to be > > > used on reboot? I would specifically exclude the possibility of this > > > reconfiguring the running domain for simplicity. > > > > I''ve only lightly tested the following, but it seemed to do what I > > expected (I used it to change memory from 512 to 1024 for a Windows VM > > on reboot). > > > > I think there might be a better name, that better reflects the fact that > > it doesn''t actually change the config right now, any ideas? > > "domain-config-override"? > > > > Thoughts on the general concept? > > This looks good to me. I''m tempted to just apply it but we can wait a > bit more for comments. > > Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>Do you want/need me to repost this? My local copy still seems to apply FWIW. Ian.
Ian Campbell writes ("Re: [Xen-devel] Re-reading domain configs on domain
restart"):> I''ve only lightly tested the following, but it seemed to do what I
> expected (I used it to change memory from 512 to 1024 for a Windows VM
> on reboot).
...> xl: provide a command to set the saved configuration for a running domain
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>